Jul 28 13:59:43.776337 ip-10-0-143-3 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 13:59:44.085099 ip-10-0-143-3 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 13:59:44.085099 ip-10-0-143-3 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 13:59:44.085099 ip-10-0-143-3 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 13:59:44.085099 ip-10-0-143-3 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 13:59:44.085099 ip-10-0-143-3 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 13:59:44.087210 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.087120    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 13:59:44.092503 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092484    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 13:59:44.092503 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092503    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092507    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092510    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092516    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092520    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092524    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092527    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092530    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092533    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092536    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092539    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092542    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092545    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092555    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092558    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092560    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092563    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092565    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092568    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092570    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 13:59:44.092571 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092573    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092576    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092579    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092581    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092585    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092588    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092590    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092593    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092595    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092598    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092600    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092602    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092605    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092607    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092611    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092614    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092617    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092620    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092623    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092625    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 13:59:44.093078 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092628    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092631    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092635    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092638    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092642    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092645    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092653    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092657    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092659    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092662    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092664    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092666    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092669    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092671    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092676    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092678    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092681    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092684    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092686    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 13:59:44.093820 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092689    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092692    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092694    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092697    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092699    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092702    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092705    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092708    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092710    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092713    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092715    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092717    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092721    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092724    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092726    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092729    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092732    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092734    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092736    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092739    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 13:59:44.094324 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092752    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092755    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092758    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092760    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092763    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092765    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092768    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092771    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092774    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092776    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092779    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092781    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092783    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092786    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092788    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092791    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092799    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092801    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 13:59:44.094808 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.092804    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 13:59:44.095684 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095667    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095686    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095690    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095693    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095698    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095702    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095705    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095708    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095711    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095714    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 13:59:44.095716 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095718    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095722    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095727    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095730    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095733    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095736    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095739    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095742    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095744    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095747    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095749    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095752    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095755    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095757    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095760    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095762    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095765    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095767    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095779    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 13:59:44.095958 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095782    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095784    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095787    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095789    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095793    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095795    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095798    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095800    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095803    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095805    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095808    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095810    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095813    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095815    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095818    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095821    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095824    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095826    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095829    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095832    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 13:59:44.096455 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095835    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095838    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095841    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095843    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095846    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095849    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095851    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095855    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095857    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095860    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095862    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095865    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095867    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095870    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095872    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095875    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095877    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095880    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095882    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095885    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 13:59:44.096945 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095888    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095890    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095892    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095895    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095898    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095900    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095903    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095906    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095910    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095913    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095915    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095918    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095920    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095923    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095926    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095929    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095932    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095934    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095937    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095939    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 13:59:44.097456 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095942    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095944    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095947    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095949    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095952    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095954    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095957    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095960    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095962    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.095965    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096374    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096379    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096382    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096385    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096388    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096391    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096394    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096397    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096399    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096403    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 13:59:44.097937 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096406    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096410    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096414    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096417    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096420    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096422    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096426    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096429    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096431    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096434    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096437    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096440    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096444    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096447    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096450    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096453    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096456    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096459    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096463    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 13:59:44.098462 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096466    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096468    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096472    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096475    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096478    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096480    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096483    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096485    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096487    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096490    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096493    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096495    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096499    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096501    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096504    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096507    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096510    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096513    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096515    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 13:59:44.098920 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096518    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096521    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096523    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096526    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096528    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096531    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096534    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096537    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096539    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096541    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096544    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096546    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096551    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096553    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096556    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096558    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096561    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096563    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096566    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096568    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 13:59:44.099499 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096571    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096573    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096576    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096578    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096581    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096584    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096587    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096589    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096591    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096594    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096596    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096599    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096601    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096604    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096607    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096609    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096611    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096614    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096617    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096619    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 13:59:44.100181 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096622    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096624    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096627    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096629    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096634    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096637    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096639    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096642    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096644    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096647    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.096650    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096734    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096744    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096757    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096763    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096769    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096775    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096782    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096787    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096791    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096794    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 13:59:44.100737 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096797    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096801    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096804    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096808    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096810    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096813    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096817    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096820    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096825    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096828    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096831    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096834    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096837    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096846    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096849    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096853    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096857    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096860    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096863    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096866    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096869    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096872    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096876    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096879    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096882    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 13:59:44.101379 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096885    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096888    2579 flags.go:64] FLAG: --enable-server="true"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096891    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096897    2579 flags.go:64] FLAG: --event-burst="100"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096900    2579 flags.go:64] FLAG: --event-qps="50"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096903    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096906    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096909    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096913    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096916    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096919    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096922    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096925    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096928    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096932    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096935    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096937    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096940    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096943    2579 flags.go:64] FLAG: --feature-gates=""
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096947    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096951    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096954    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096958    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096961    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096964    2579 flags.go:64] FLAG: --help="false"
Jul 28 13:59:44.102067 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096967    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096970    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096973    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096976    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096980    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096983    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096986    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096989    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096992    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096994    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.096998    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097001    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097004    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097007    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097010    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097027    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097030    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097033    2579 flags.go:64] FLAG: --lock-file=""
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097035    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097038    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097041    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097047    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097050    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097053    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 13:59:44.102801 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097056    2579 flags.go:64] FLAG: --logging-format="text"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097059    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097062    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097065    2579 flags.go:64] FLAG: --manifest-url=""
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097068    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097073    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097076    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097081    2579 flags.go:64] FLAG: --max-pods="110"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097084    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097087    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097090    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097093    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097096    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097099    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097102    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097109    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097112    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097115    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097119    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097122    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097128    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097131    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097134    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097137    2579 flags.go:64] FLAG: --port="10250"
Jul 28 13:59:44.103486 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097140    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097142    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d231dd8a01c3d505"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097146    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097149    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097152    2579 flags.go:64] FLAG: --register-node="true"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097154    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097158    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097162    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097165    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097167    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097171    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097174    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097177    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097180    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097191    2579 flags.go:64] FLAG: --runonce="false"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097194    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097198    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097201    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097203    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097206    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097209    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097213    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097216    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097219    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097222    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 13:59:44.104504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097225    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097229    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097232    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097235    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097238    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097244    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097247    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097249    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097254    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097257    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097260    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097262    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097265    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097268    2579 flags.go:64] FLAG: --v="2"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097272    2579 flags.go:64] FLAG: --version="false"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097277    2579 flags.go:64] FLAG: --vmodule=""
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097281    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.097284    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097403    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097407    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097410    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097414    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097417    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 13:59:44.105723 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097419    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097422    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097425    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097427    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097430    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097432    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097435    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097437    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097440    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097443    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097446    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097449    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097452    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097455    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097457    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097460    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097463    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097465    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097468    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 13:59:44.106779 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097470    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097475    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097479    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097482    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097485    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097488    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097491    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097494    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097496    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097499    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097501    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097505    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097507    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097510    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097512    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097515    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097517    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097520    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097522    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 13:59:44.107509 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097524    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097527    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097529    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097532    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097536    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097539    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097541    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097544    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097547    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097549    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097551    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097555    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097559    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097561    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097564    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097566    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097569    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097571    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097574    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097576    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 13:59:44.108142 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097579    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097582    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097584    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097586    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097589    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097592    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097595    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097597    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097599    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097602    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097604    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097607    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097609    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097612    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097616    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097619    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097623    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097626    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097629    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 13:59:44.108952 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097632    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097635    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097637    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097640    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097642    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097645    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097647    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097650    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097652    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097655    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097657    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097660    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097662    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097665    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097668    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097671    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 13:59:44.109620 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.097673    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.098156    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.105812    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.105832    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.105859    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.105870    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.105955    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.105962    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.105968    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.105973    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.105978    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.105983    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.105988    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.105992    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 13:59:44.110229 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.105997    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106001    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106005    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106010    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106036    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106041    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106046    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106050    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106054    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106059    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106063    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106067    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106071    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106075    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106079    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106084    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106088    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106092    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106097    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106101    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 13:59:44.110583 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106105    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106111    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106116    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106120    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106124    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106128    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106133    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106137    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106141    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106145    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106149    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106154    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106158    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106162    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106166    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106170    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106174    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106178    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106182    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 13:59:44.111125 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106186    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106193    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106200    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106205    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106211    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106215    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106220    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106225    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106231    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106237    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106241    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106246    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106250    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106256    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106261    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106266    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106271    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106275    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106279    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 13:59:44.111655 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106283    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106288    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106294    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106300    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106305    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106309    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106313    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106318    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106322    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106326    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106331    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106335    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106339    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106344    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106348    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106353    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106357    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106361    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106365    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106369    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 13:59:44.112199 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106373    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106377    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106381    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106386    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106390    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106395    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106399    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106404    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106408    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106413    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106418    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106422    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106426    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.106435    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106618    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 13:59:44.112712 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106626    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106631    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106635    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106640    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106644    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106648    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106652    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106656    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106660    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106664    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106669    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106674    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106678    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106682    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106686    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106691    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106695    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106700    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106704    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106708    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 13:59:44.113202 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106712    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106717    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106721    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106726    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106730    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106734    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106739    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106744    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106748    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106752    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106756    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106761    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106765    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106769    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106773    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106777    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106781    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106785    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106788    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106792    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 13:59:44.114006 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106796    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106800    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106804    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106809    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106813    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106817    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106822    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106826    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106830    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106835    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106839    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106843    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106847    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106851    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106855    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106860    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106864    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106869    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106873    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106877    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 13:59:44.114535 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106882    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106887    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106891    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106895    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106899    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106904    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106908    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106913    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106917    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106921    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106925    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106932    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106937    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106941    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106945    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106950    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106954    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106958    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106963    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106967    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 13:59:44.115065 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106971    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106975    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106979    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106984    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106988    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106992    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.106996    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107000    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107005    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107009    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107032    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107039    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107044    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107050    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107055    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107061    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107067    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 13:59:44.115596 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:44.107072    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 13:59:44.116306 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.107081    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 13:59:44.116306 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.107247    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 13:59:44.116306 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.109934    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 13:59:44.116306 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.111194    2579 server.go:1019] "Starting client certificate rotation"
Jul 28 13:59:44.116306 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.111303    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 13:59:44.116306 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.111711    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 13:59:44.116306 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.113653    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 13:59:44.118348 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.118322    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 13:59:44.127334 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.127314    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 28 13:59:44.129325 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.129304    2579 log.go:25] "Validated CRI v1 image API"
Jul 28 13:59:44.130584 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.130569    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 13:59:44.132606 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.132587    2579 fs.go:135] Filesystem UUIDs: map[15f4b4fb-a476-48cc-af2f-eda08d60e7d7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c9dae155-9152-4a48-9462-65fc54baad1d:/dev/nvme0n1p4]
Jul 28 13:59:44.132672 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.132605    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 13:59:44.136783 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.136765    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 13:59:44.137961 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.137855    2579 manager.go:216] Machine: {Timestamp:2026-07-28 13:59:44.13655914 +0000 UTC m=+0.263949821 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3089817 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22cf1285b232f4f3ecaf1396f9bf06 SystemUUID:ec22cf12-85b2-32f4-f3ec-af1396f9bf06 BootID:f8c58de3-baf0-4db3-a128-1267eb0524b9 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:37:64:16:e4:b1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:37:64:16:e4:b1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:dd:13:65:ce:1b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 13:59:44.137961 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.137956    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 13:59:44.138087 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.138053    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 13:59:44.140582 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.140558    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 13:59:44.140721 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.140585    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-3.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 13:59:44.141259 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.141247    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 13:59:44.141297 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.141262    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 13:59:44.141297 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.141270    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 13:59:44.141297 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.141284    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 13:59:44.142104 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.142095    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 13:59:44.144745 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.144733    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 13:59:44.145060 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.145049    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 13:59:44.145101 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.145082    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 13:59:44.145129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.145103    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 13:59:44.145129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.145124    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 13:59:44.145178 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.145136    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 28 13:59:44.145178 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.145153    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 13:59:44.145517 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.145505    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 13:59:44.145552 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.145522    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 13:59:44.147346 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.147332    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 13:59:44.148995 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.148983    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 13:59:44.149056 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.149027    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 13:59:44.150382 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150371    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 13:59:44.150449 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150391    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 13:59:44.150449 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150397    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 13:59:44.150449 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150403    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 13:59:44.150449 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150413    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 13:59:44.150449 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150420    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 13:59:44.150449 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150430    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 13:59:44.150449 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150438    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 13:59:44.150449 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150446    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 13:59:44.150449 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150454    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 13:59:44.150765 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150479    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 13:59:44.150765 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.150489    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 13:59:44.151679 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.151669    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 13:59:44.151679 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.151680    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 13:59:44.154975 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.154959    2579 server.go:1268] "Started kubelet"
Jul 28 13:59:44.158178 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.155071    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 13:59:44.158178 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.157845    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 13:59:44.158178 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.158077    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 13:59:44.158581 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.158562    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-3.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 13:59:44.158912 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.158875    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-3.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 13:59:44.158912 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.158832    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 13:59:44.160085 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.159482    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 13:59:44.159622 ip-10-0-143-3 systemd[1]: Started Kubernetes Kubelet.
Jul 28 13:59:44.161429 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.161385    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 13:59:44.162967 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.162947    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 13:59:44.165050 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.164204    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-3.ec2.internal.18c678daf7d21ed6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-3.ec2.internal,UID:ip-10-0-143-3.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-3.ec2.internal,},FirstTimestamp:2026-07-28 13:59:44.154926806 +0000 UTC m=+0.282317488,LastTimestamp:2026-07-28 13:59:44.154926806 +0000 UTC m=+0.282317488,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-3.ec2.internal,}"
Jul 28 13:59:44.167249 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.167229    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 13:59:44.167331 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.167230    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 13:59:44.167403 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.167346    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:44.167806 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.167734    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 13:59:44.167806 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.167736    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 13:59:44.167944 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.167819    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 13:59:44.167944 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.167940    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 13:59:44.168075 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.167953    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 13:59:44.168267 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.168252    2579 factory.go:55] Registering systemd factory
Jul 28 13:59:44.168267 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.168271    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 28 13:59:44.168548 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.168530    2579 factory.go:153] Registering CRI-O factory
Jul 28 13:59:44.168548 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.168544    2579 factory.go:223] Registration of the crio container factory successfully
Jul 28 13:59:44.168656 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.168598    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 13:59:44.168656 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.168624    2579 factory.go:103] Registering Raw factory
Jul 28 13:59:44.168656 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.168640    2579 manager.go:1196] Started watching for new ooms in manager
Jul 28 13:59:44.169100 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.169085    2579 manager.go:349] Starting recovery of all containers
Jul 28 13:59:44.172414 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.172366    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 13:59:44.172414 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.172365    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-3.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 13:59:44.182095 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.182005    2579 manager.go:354] Recovery completed
Jul 28 13:59:44.187068 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.187049    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 13:59:44.189243 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.189109    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 13:59:44.189243 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.189169    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 13:59:44.189243 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.189190    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientPID"
Jul 28 13:59:44.189608 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.189590    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tqcfd"
Jul 28 13:59:44.189814 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.189798    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 13:59:44.189814 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.189812    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 13:59:44.189963 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.189827    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 13:59:44.189963 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.189802    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 13:59:44.190702 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.190684    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 13:59:44.190769 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.190723    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 13:59:44.190769 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.190739    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientPID"
Jul 28 13:59:44.191087 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.190995    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-3.ec2.internal.18c678daf9dc2ae0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-3.ec2.internal,UID:ip-10-0-143-3.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-3.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-3.ec2.internal,},FirstTimestamp:2026-07-28 13:59:44.18913968 +0000 UTC m=+0.316530367,LastTimestamp:2026-07-28 13:59:44.18913968 +0000 UTC m=+0.316530367,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-3.ec2.internal,}"
Jul 28 13:59:44.192448 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.192436    2579 policy_none.go:49] "None policy: Start"
Jul 28 13:59:44.192489 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.192454    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 13:59:44.192489 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.192464    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 13:59:44.194432 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.194421    2579 policy_none.go:47] "Start"
Jul 28 13:59:44.201453 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.201431    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tqcfd"
Jul 28 13:59:44.202294 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.202186    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-3.ec2.internal.18c678daf9dcb5f3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-3.ec2.internal,UID:ip-10-0-143-3.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-3.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-3.ec2.internal,},FirstTimestamp:2026-07-28 13:59:44.189175283 +0000 UTC m=+0.316565966,LastTimestamp:2026-07-28 13:59:44.189175283 +0000 UTC m=+0.316565966,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-3.ec2.internal,}"
Jul 28 13:59:44.230679 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.230656    2579 manager.go:337] "Starting Device Plugin manager"
Jul 28 13:59:44.230807 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.230702    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 13:59:44.230807 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.230713    2579 server.go:87] "Starting device plugin registration server"
Jul 28 13:59:44.231073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.231056    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 13:59:44.231149 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.231075    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 13:59:44.231202 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.231182    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 13:59:44.231306 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.231291    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 13:59:44.231306 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.231305    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 13:59:44.231872 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.231848    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 13:59:44.231961 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.231904    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:44.331205 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.331181    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 13:59:44.333440 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.332202    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 13:59:44.333440 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.332233    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 13:59:44.333440 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.332243    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientPID"
Jul 28 13:59:44.333440 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.332270    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.333722 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.333700    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 13:59:44.335204 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.335162    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 13:59:44.335253 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.335210    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 13:59:44.335253 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.335235    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 13:59:44.335310 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.335275    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 13:59:44.337715 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.337693    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 13:59:44.343604 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.343587    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.343684 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.343610    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-3.ec2.internal\": node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:44.399526 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.399492    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:44.435567 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.435535    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal"]
Jul 28 13:59:44.435654 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.435625    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 13:59:44.437465 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.437447    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 13:59:44.437573 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.437479    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 13:59:44.437573 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.437491    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientPID"
Jul 28 13:59:44.439881 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.439869    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 13:59:44.440042 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.440006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.440081 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.440060    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 13:59:44.440637 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.440621    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 13:59:44.440709 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.440646    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 13:59:44.440709 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.440657    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientPID"
Jul 28 13:59:44.440709 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.440622    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 13:59:44.440845 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.440733    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 13:59:44.440845 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.440752    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientPID"
Jul 28 13:59:44.443081 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.443064    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.443171 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.443095    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 13:59:44.443937 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.443920    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 13:59:44.444003 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.443949    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 13:59:44.444003 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.443959    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeHasSufficientPID"
Jul 28 13:59:44.459585 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.459566    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-3.ec2.internal\" not found" node="ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.463825 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.463810    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-3.ec2.internal\" not found" node="ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.469604 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.469580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2dbdcd5940fe0b083a0d80c7c2740683-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal\" (UID: \"2dbdcd5940fe0b083a0d80c7c2740683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.469683 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.469611    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dbdcd5940fe0b083a0d80c7c2740683-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal\" (UID: \"2dbdcd5940fe0b083a0d80c7c2740683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.469683 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.469632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6180e30e1977ed4d8d23fa6e6c33c1ed-config\") pod \"kube-apiserver-proxy-ip-10-0-143-3.ec2.internal\" (UID: \"6180e30e1977ed4d8d23fa6e6c33c1ed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.500401 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.500379    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:44.570653 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.570623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2dbdcd5940fe0b083a0d80c7c2740683-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal\" (UID: \"2dbdcd5940fe0b083a0d80c7c2740683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.570653 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.570656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dbdcd5940fe0b083a0d80c7c2740683-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal\" (UID: \"2dbdcd5940fe0b083a0d80c7c2740683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.570843 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.570679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6180e30e1977ed4d8d23fa6e6c33c1ed-config\") pod \"kube-apiserver-proxy-ip-10-0-143-3.ec2.internal\" (UID: \"6180e30e1977ed4d8d23fa6e6c33c1ed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.570843 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.570736    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6180e30e1977ed4d8d23fa6e6c33c1ed-config\") pod \"kube-apiserver-proxy-ip-10-0-143-3.ec2.internal\" (UID: \"6180e30e1977ed4d8d23fa6e6c33c1ed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.570843 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.570746    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2dbdcd5940fe0b083a0d80c7c2740683-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal\" (UID: \"2dbdcd5940fe0b083a0d80c7c2740683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.570843 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.570801    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dbdcd5940fe0b083a0d80c7c2740683-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal\" (UID: \"2dbdcd5940fe0b083a0d80c7c2740683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.600805 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.600738    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:44.701600 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.701556    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:44.761762 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.761734    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.766851 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:44.766706    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:44.801858 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.801832    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:44.902448 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:44.902342    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:45.002874 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:45.002838    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:45.103337 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:45.103300    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:45.111471 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.111447    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 13:59:45.111604 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.111588    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 13:59:45.194064 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.194011    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 13:59:45.203730 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:45.203688    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-3.ec2.internal\" not found"
Jul 28 13:59:45.206848 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.206816    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 13:54:45 +0000 UTC" deadline="2028-04-30 03:38:25.40010514 +0000 UTC"
Jul 28 13:59:45.206848 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.206846    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15397h38m40.193262686s"
Jul 28 13:59:45.214085 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.214064    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 13:59:45.259847 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.259824    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 13:59:45.267960 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.267497    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:45.278815 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.278796    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 13:59:45.279693 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.279596    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal"
Jul 28 13:59:45.285859 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:45.285819    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dbdcd5940fe0b083a0d80c7c2740683.slice/crio-f0d9ff1c06e26ebf5bdb6f044bc4081c4c78de8c3fb187e5ec783a04c1a76007 WatchSource:0}: Error finding container f0d9ff1c06e26ebf5bdb6f044bc4081c4c78de8c3fb187e5ec783a04c1a76007: Status 404 returned error can't find the container with id f0d9ff1c06e26ebf5bdb6f044bc4081c4c78de8c3fb187e5ec783a04c1a76007
Jul 28 13:59:45.286208 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:45.286183    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6180e30e1977ed4d8d23fa6e6c33c1ed.slice/crio-61c3027c1f2a168f204f7425945e80c4d1b69e1b913c6ba48226f10a38720f85 WatchSource:0}: Error finding container 61c3027c1f2a168f204f7425945e80c4d1b69e1b913c6ba48226f10a38720f85: Status 404 returned error can't find the container with id 61c3027c1f2a168f204f7425945e80c4d1b69e1b913c6ba48226f10a38720f85
Jul 28 13:59:45.289896 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.289881    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:59:45.299154 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.299134    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 13:59:45.339330 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.339282    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal" event={"ID":"6180e30e1977ed4d8d23fa6e6c33c1ed","Type":"ContainerStarted","Data":"61c3027c1f2a168f204f7425945e80c4d1b69e1b913c6ba48226f10a38720f85"}
Jul 28 13:59:45.340423 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:45.340401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal" event={"ID":"2dbdcd5940fe0b083a0d80c7c2740683","Type":"ContainerStarted","Data":"f0d9ff1c06e26ebf5bdb6f044bc4081c4c78de8c3fb187e5ec783a04c1a76007"}
Jul 28 13:59:46.146457 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.146425    2579 apiserver.go:52] "Watching apiserver"
Jul 28 13:59:46.156768 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.156728    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 13:59:46.157890 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.157858    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-jgqhc","kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal","openshift-cluster-node-tuning-operator/tuned-kphgf","openshift-dns/node-resolver-zfd6l","openshift-image-registry/node-ca-9df2t","openshift-multus/multus-additional-cni-plugins-w7lvd","openshift-multus/multus-vs5zj","openshift-multus/network-metrics-daemon-8ltnr","kube-system/global-pull-secret-syncer-c75dk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal","openshift-network-diagnostics/network-check-target-shngl","openshift-network-operator/iptables-alerter-2gpf4","openshift-ovn-kubernetes/ovnkube-node-m2b9r"]
Jul 28 13:59:46.160660 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.160637    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.163413 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.162949    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.163713 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.163688    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 13:59:46.163825 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.163771    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 13:59:46.163825 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.163813    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 13:59:46.163976 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.163956    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9xbt9\""
Jul 28 13:59:46.164084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.164023    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 13:59:46.167555 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.167537    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 13:59:46.167907 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.167885    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 13:59:46.168007 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.167885    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 13:59:46.168098 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.168082    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 13:59:46.168216 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.168197    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vt9l8\""
Jul 28 13:59:46.168677 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.168660    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.171198 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.170887    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 13:59:46.171198 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.170967    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 13:59:46.171198 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.170976    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h4wkp\""
Jul 28 13:59:46.171198 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.171173    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.173441 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.173422    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.173582 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.173563    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 13:59:46.173966 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.173783    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 13:59:46.173966 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.173873    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qckb8\""
Jul 28 13:59:46.175916 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.175895    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jr78q\""
Jul 28 13:59:46.176094 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.175923    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 13:59:46.176094 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.175971    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 13:59:46.176094 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.175928    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 13:59:46.176353 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.176313    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.178485 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.178463    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.178585 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.178550    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 13:59:46.178585 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.178559    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zfsc2\""
Jul 28 13:59:46.178692 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.178541    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 13:59:46.179059 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.178873    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 13:59:46.179304 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.179284    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 13:59:46.180380 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-kubernetes\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.180490 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180387    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-sys\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.180490 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180420    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-os-release\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.180490 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-multus-socket-dir-parent\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.180490 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-run-netns\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.180702 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-var-lib-cni-multus\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.180702 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/846645a8-4f68-42e5-bfcb-abed04caea71-multus-daemon-config\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.180702 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-run-multus-certs\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.180702 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180624    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-tuned\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.180702 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180648    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-system-cni-dir\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.180702 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-hostroot\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.180702 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180692    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htdwx\" (UniqueName: \"kubernetes.io/projected/846645a8-4f68-42e5-bfcb-abed04caea71-kube-api-access-htdwx\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-run\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180745    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-modprobe-d\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-lib-modules\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180838    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a75b1cb7-e545-42f1-9f57-6c342726495d-tmp\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx92l\" (UniqueName: \"kubernetes.io/projected/a75b1cb7-e545-42f1-9f57-6c342726495d-kube-api-access-hx92l\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-multus-cni-dir\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-etc-kubernetes\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-sysconfig\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-cnibin\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.180994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms2th\" (UniqueName: \"kubernetes.io/projected/4622aa17-12b9-40ab-9430-e7e2101b8f45-kube-api-access-ms2th\") pod \"iptables-alerter-2gpf4\" (UID: \"4622aa17-12b9-40ab-9430-e7e2101b8f45\") " pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.181084 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181028    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-sysctl-d\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/846645a8-4f68-42e5-bfcb-abed04caea71-cni-binary-copy\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181128    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-var-lib-cni-bin\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181168    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-multus-conf-dir\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4622aa17-12b9-40ab-9430-e7e2101b8f45-host-slash\") pod \"iptables-alerter-2gpf4\" (UID: \"4622aa17-12b9-40ab-9430-e7e2101b8f45\") " pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-sysctl-conf\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181254    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-host\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-run-k8s-cni-cncf-io\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181296    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-var-lib-kubelet\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181338    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-systemd\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-var-lib-kubelet\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181408    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4622aa17-12b9-40ab-9430-e7e2101b8f45-iptables-alerter-script\") pod \"iptables-alerter-2gpf4\" (UID: \"4622aa17-12b9-40ab-9430-e7e2101b8f45\") " pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.181619 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.181542    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 13:59:46.183905 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.183881    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 13:59:46.184011 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.183908    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kqh4m\""
Jul 28 13:59:46.184160 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.184135    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 13:59:46.184783 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.184366    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.187177 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.187153    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.188255 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.188231    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gg45c\""
Jul 28 13:59:46.188348 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.188282    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 13:59:46.188489 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.188473    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 13:59:46.188557 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.188534    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 13:59:46.188622 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.188236    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 13:59:46.188741 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.188726    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 13:59:46.188783 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.188494    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 13:59:46.189615 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.189532    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 13:59:46.189736 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.189716    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5nkqh\""
Jul 28 13:59:46.189811 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.189743    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 13:59:46.189853 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.189532    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 13:59:46.192606 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.190145    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:46.192606 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.190197    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 13:59:46.195203 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.193243    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:46.195203 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.193455    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 13:59:46.221463 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.221414    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vsmnj"
Jul 28 13:59:46.229524 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.229433    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vsmnj"
Jul 28 13:59:46.269618 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.269583    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 13:59:46.281997 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.281962    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-node-log\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.281997 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.281997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-cni-netd\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.282249 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282018    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/43225ca8-b2b9-42a1-b642-a59a089dd46d-ovn-node-metrics-cert\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.282249 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-kubernetes\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.282249 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-os-release\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.282249 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282182    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-multus-socket-dir-parent\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.282249 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-run-multus-certs\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.282249 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282214    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-os-release\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.282249 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjzck\" (UniqueName: \"kubernetes.io/projected/d510aeee-4e32-4319-a8c6-a47ba4ae9205-kube-api-access-zjzck\") pod \"node-ca-9df2t\" (UID: \"d510aeee-4e32-4319-a8c6-a47ba4ae9205\") " pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282271    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-multus-socket-dir-parent\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282285    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-kubernetes\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282311    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-run-multus-certs\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282330    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxv27\" (UniqueName: \"kubernetes.io/projected/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-kube-api-access-gxv27\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-tuned\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282396    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-system-cni-dir\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdwx\" (UniqueName: \"kubernetes.io/projected/846645a8-4f68-42e5-bfcb-abed04caea71-kube-api-access-htdwx\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d510aeee-4e32-4319-a8c6-a47ba4ae9205-host\") pod \"node-ca-9df2t\" (UID: \"d510aeee-4e32-4319-a8c6-a47ba4ae9205\") " pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282472    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-run\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282509    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-system-cni-dir\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282512    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-etc-openvswitch\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.282616 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-modprobe-d\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282631    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-run\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-modprobe-d\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-lib-modules\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a75b1cb7-e545-42f1-9f57-6c342726495d-tmp\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-multus-cni-dir\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282768    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d510aeee-4e32-4319-a8c6-a47ba4ae9205-serviceca\") pod \"node-ca-9df2t\" (UID: \"d510aeee-4e32-4319-a8c6-a47ba4ae9205\") " pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-systemd-units\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282828    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-run-openvswitch\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282850    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-lib-modules\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282911    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-multus-cni-dir\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282916    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xft9h\" (UniqueName: \"kubernetes.io/projected/43225ca8-b2b9-42a1-b642-a59a089dd46d-kube-api-access-xft9h\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-sysconfig\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-cnibin\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.282999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-sysconfig\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283005    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-sys-fs\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283061    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-cnibin\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.283129 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/43225ca8-b2b9-42a1-b642-a59a089dd46d-ovnkube-script-lib\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283133    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283161    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283190    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/846645a8-4f68-42e5-bfcb-abed04caea71-cni-binary-copy\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-var-lib-cni-bin\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-multus-conf-dir\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68fa69ca-ef93-41a4-9b62-14e2804b3e26-hosts-file\") pod \"node-resolver-zfd6l\" (UID: \"68fa69ca-ef93-41a4-9b62-14e2804b3e26\") " pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctnjh\" (UniqueName: \"kubernetes.io/projected/68fa69ca-ef93-41a4-9b62-14e2804b3e26-kube-api-access-ctnjh\") pod \"node-resolver-zfd6l\" (UID: \"68fa69ca-ef93-41a4-9b62-14e2804b3e26\") " pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283335    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-var-lib-cni-bin\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283353    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-multus-conf-dir\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-dbus\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283406    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-sysctl-conf\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-var-lib-kubelet\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283521    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-system-cni-dir\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-sysctl-conf\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.284240 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f4b440f-f03f-4c5e-ad0b-0935b299d316-konnectivity-ca\") pod \"konnectivity-agent-jgqhc\" (UID: \"2f4b440f-f03f-4c5e-ad0b-0935b299d316\") " pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283639    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-var-lib-kubelet\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-run-netns\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-var-lib-kubelet\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283781    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-var-lib-kubelet\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283803    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4622aa17-12b9-40ab-9430-e7e2101b8f45-iptables-alerter-script\") pod \"iptables-alerter-2gpf4\" (UID: \"4622aa17-12b9-40ab-9430-e7e2101b8f45\") " pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283832    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f4b440f-f03f-4c5e-ad0b-0935b299d316-agent-certs\") pod \"konnectivity-agent-jgqhc\" (UID: \"2f4b440f-f03f-4c5e-ad0b-0935b299d316\") " pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-run-systemd\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-run-ovn\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283904    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-cni-bin\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/43225ca8-b2b9-42a1-b642-a59a089dd46d-env-overrides\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283954    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-sys\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.283980    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-run-netns\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-var-lib-cni-multus\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/846645a8-4f68-42e5-bfcb-abed04caea71-multus-daemon-config\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284126    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-var-lib-openvswitch\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.285073 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-sys\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-run-netns\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-hostroot\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-registration-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284336    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284432    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrsxq\" (UniqueName: \"kubernetes.io/projected/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-kube-api-access-lrsxq\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-run-ovn-kubernetes\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx92l\" (UniqueName: \"kubernetes.io/projected/a75b1cb7-e545-42f1-9f57-6c342726495d-kube-api-access-hx92l\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4622aa17-12b9-40ab-9430-e7e2101b8f45-iptables-alerter-script\") pod \"iptables-alerter-2gpf4\" (UID: \"4622aa17-12b9-40ab-9430-e7e2101b8f45\") " pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284537    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-etc-kubernetes\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284549    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/846645a8-4f68-42e5-bfcb-abed04caea71-multus-daemon-config\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-cni-binary-copy\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284591    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-hostroot\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-log-socket\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284627    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-var-lib-cni-multus\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284654    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/43225ca8-b2b9-42a1-b642-a59a089dd46d-ovnkube-config\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.285897 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms2th\" (UniqueName: \"kubernetes.io/projected/4622aa17-12b9-40ab-9430-e7e2101b8f45-kube-api-access-ms2th\") pod \"iptables-alerter-2gpf4\" (UID: \"4622aa17-12b9-40ab-9430-e7e2101b8f45\") " pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-etc-kubernetes\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284747    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg8fl\" (UniqueName: \"kubernetes.io/projected/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-kube-api-access-kg8fl\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284811    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/846645a8-4f68-42e5-bfcb-abed04caea71-cni-binary-copy\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-sysctl-d\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4622aa17-12b9-40ab-9430-e7e2101b8f45-host-slash\") pod \"iptables-alerter-2gpf4\" (UID: \"4622aa17-12b9-40ab-9430-e7e2101b8f45\") " pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.284979    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285062    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4622aa17-12b9-40ab-9430-e7e2101b8f45-host-slash\") pod \"iptables-alerter-2gpf4\" (UID: \"4622aa17-12b9-40ab-9430-e7e2101b8f45\") " pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-host\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285121    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-run-k8s-cni-cncf-io\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285157    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-sysctl-d\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285201    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-host\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285205    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68fa69ca-ef93-41a4-9b62-14e2804b3e26-tmp-dir\") pod \"node-resolver-zfd6l\" (UID: \"68fa69ca-ef93-41a4-9b62-14e2804b3e26\") " pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285259    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-kubelet-config\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-socket-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/846645a8-4f68-42e5-bfcb-abed04caea71-host-run-k8s-cni-cncf-io\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.286647 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-cnibin\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.287299 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285336    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-os-release\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.287299 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-slash\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.287299 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-systemd\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.287299 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-device-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.287299 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285446    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-kubelet\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.287299 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.285455    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-systemd\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.287299 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.286506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a75b1cb7-e545-42f1-9f57-6c342726495d-etc-tuned\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.287299 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.286592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a75b1cb7-e545-42f1-9f57-6c342726495d-tmp\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.301679 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.301593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htdwx\" (UniqueName: \"kubernetes.io/projected/846645a8-4f68-42e5-bfcb-abed04caea71-kube-api-access-htdwx\") pod \"multus-vs5zj\" (UID: \"846645a8-4f68-42e5-bfcb-abed04caea71\") " pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.301679 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.301625    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms2th\" (UniqueName: \"kubernetes.io/projected/4622aa17-12b9-40ab-9430-e7e2101b8f45-kube-api-access-ms2th\") pod \"iptables-alerter-2gpf4\" (UID: \"4622aa17-12b9-40ab-9430-e7e2101b8f45\") " pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.301679 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.301616    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx92l\" (UniqueName: \"kubernetes.io/projected/a75b1cb7-e545-42f1-9f57-6c342726495d-kube-api-access-hx92l\") pod \"tuned-kphgf\" (UID: \"a75b1cb7-e545-42f1-9f57-6c342726495d\") " pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.353454 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.353417    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.386809    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-etc-openvswitch\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.386651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-etc-openvswitch\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387156    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d510aeee-4e32-4319-a8c6-a47ba4ae9205-serviceca\") pod \"node-ca-9df2t\" (UID: \"d510aeee-4e32-4319-a8c6-a47ba4ae9205\") " pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387182    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-systemd-units\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387208    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-run-openvswitch\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xft9h\" (UniqueName: \"kubernetes.io/projected/43225ca8-b2b9-42a1-b642-a59a089dd46d-kube-api-access-xft9h\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-sys-fs\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/43225ca8-b2b9-42a1-b642-a59a089dd46d-ovnkube-script-lib\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387321    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.387590    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.387685    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs podName:66bf6b5d-9493-4aaf-aa4f-2f3788bf51be nodeName:}" failed. No retries permitted until 2026-07-28 13:59:46.887639828 +0000 UTC m=+3.015030499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs") pod "network-metrics-daemon-8ltnr" (UID: "66bf6b5d-9493-4aaf-aa4f-2f3788bf51be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387696    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-systemd-units\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387749    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d510aeee-4e32-4319-a8c6-a47ba4ae9205-serviceca\") pod \"node-ca-9df2t\" (UID: \"d510aeee-4e32-4319-a8c6-a47ba4ae9205\") " pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387758    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-run-openvswitch\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68fa69ca-ef93-41a4-9b62-14e2804b3e26-hosts-file\") pod \"node-resolver-zfd6l\" (UID: \"68fa69ca-ef93-41a4-9b62-14e2804b3e26\") " pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.388223 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-sys-fs\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387829    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctnjh\" (UniqueName: \"kubernetes.io/projected/68fa69ca-ef93-41a4-9b62-14e2804b3e26-kube-api-access-ctnjh\") pod \"node-resolver-zfd6l\" (UID: \"68fa69ca-ef93-41a4-9b62-14e2804b3e26\") " pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387855    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-dbus\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387879    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387938    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-system-cni-dir\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387944    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/43225ca8-b2b9-42a1-b642-a59a089dd46d-ovnkube-script-lib\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68fa69ca-ef93-41a4-9b62-14e2804b3e26-hosts-file\") pod \"node-resolver-zfd6l\" (UID: \"68fa69ca-ef93-41a4-9b62-14e2804b3e26\") " pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.387987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f4b440f-f03f-4c5e-ad0b-0935b299d316-konnectivity-ca\") pod \"konnectivity-agent-jgqhc\" (UID: \"2f4b440f-f03f-4c5e-ad0b-0935b299d316\") " pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-run-netns\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f4b440f-f03f-4c5e-ad0b-0935b299d316-agent-certs\") pod \"konnectivity-agent-jgqhc\" (UID: \"2f4b440f-f03f-4c5e-ad0b-0935b299d316\") " pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.388057    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388067    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-dbus\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388109    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-run-systemd\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388144    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-run-netns\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388188    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-system-cni-dir\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.390365 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.388791    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret podName:e2f0a8fb-0a74-4e58-8195-47b0ac92551b nodeName:}" failed. No retries permitted until 2026-07-28 13:59:46.888771785 +0000 UTC m=+3.016162465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret") pod "global-pull-secret-syncer-c75dk" (UID: "e2f0a8fb-0a74-4e58-8195-47b0ac92551b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-run-systemd\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-run-ovn\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-cni-bin\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/43225ca8-b2b9-42a1-b642-a59a089dd46d-env-overrides\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-var-lib-openvswitch\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388951    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-registration-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.388977    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrsxq\" (UniqueName: \"kubernetes.io/projected/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-kube-api-access-lrsxq\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389077    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-run-ovn-kubernetes\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-cni-binary-copy\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389144    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-var-lib-openvswitch\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-log-socket\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389184    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f4b440f-f03f-4c5e-ad0b-0935b299d316-konnectivity-ca\") pod \"konnectivity-agent-jgqhc\" (UID: \"2f4b440f-f03f-4c5e-ad0b-0935b299d316\") " pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 13:59:46.391188 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-log-socket\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/43225ca8-b2b9-42a1-b642-a59a089dd46d-ovnkube-config\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389266    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg8fl\" (UniqueName: \"kubernetes.io/projected/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-kube-api-access-kg8fl\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389269    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-run-ovn-kubernetes\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389300    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389362    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68fa69ca-ef93-41a4-9b62-14e2804b3e26-tmp-dir\") pod \"node-resolver-zfd6l\" (UID: \"68fa69ca-ef93-41a4-9b62-14e2804b3e26\") " pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-kubelet-config\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389414    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-socket-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-cnibin\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-os-release\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-slash\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389519    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/43225ca8-b2b9-42a1-b642-a59a089dd46d-env-overrides\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389538    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-run-ovn\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-device-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389577    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-device-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.391822 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-registration-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-kubelet\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389749    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-cni-bin\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389584    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-kubelet\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389791    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-node-log\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-cni-netd\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389829    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-node-log\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/43225ca8-b2b9-42a1-b642-a59a089dd46d-ovn-node-metrics-cert\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjzck\" (UniqueName: \"kubernetes.io/projected/d510aeee-4e32-4319-a8c6-a47ba4ae9205-kube-api-access-zjzck\") pod \"node-ca-9df2t\" (UID: \"d510aeee-4e32-4319-a8c6-a47ba4ae9205\") " pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389903    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxv27\" (UniqueName: \"kubernetes.io/projected/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-kube-api-access-gxv27\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389927    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d510aeee-4e32-4319-a8c6-a47ba4ae9205-host\") pod \"node-ca-9df2t\" (UID: \"d510aeee-4e32-4319-a8c6-a47ba4ae9205\") " pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389932    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-socket-dir\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.389977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d510aeee-4e32-4319-a8c6-a47ba4ae9205-host\") pod \"node-ca-9df2t\" (UID: \"d510aeee-4e32-4319-a8c6-a47ba4ae9205\") " pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.390003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-cni-binary-copy\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.390064    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-cni-netd\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.392525 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.390125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.393109 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.390132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-cnibin\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.393109 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.390125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-kubelet-config\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.393109 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.390178    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43225ca8-b2b9-42a1-b642-a59a089dd46d-host-slash\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.393109 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.390197    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-os-release\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.393109 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.390198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/43225ca8-b2b9-42a1-b642-a59a089dd46d-ovnkube-config\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.393109 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.390704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68fa69ca-ef93-41a4-9b62-14e2804b3e26-tmp-dir\") pod \"node-resolver-zfd6l\" (UID: \"68fa69ca-ef93-41a4-9b62-14e2804b3e26\") " pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.393109 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.391438    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f4b440f-f03f-4c5e-ad0b-0935b299d316-agent-certs\") pod \"konnectivity-agent-jgqhc\" (UID: \"2f4b440f-f03f-4c5e-ad0b-0935b299d316\") " pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 13:59:46.393109 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.392669    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/43225ca8-b2b9-42a1-b642-a59a089dd46d-ovn-node-metrics-cert\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.404417 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.404307    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 13:59:46.404417 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.404341    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 13:59:46.404417 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.404359    2579 projected.go:196] Error preparing data for projected volume kube-api-access-n4knh for pod openshift-network-diagnostics/network-check-target-shngl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:46.404771 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.404433    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh podName:7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc nodeName:}" failed. No retries permitted until 2026-07-28 13:59:46.904412062 +0000 UTC m=+3.031802747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n4knh" (UniqueName: "kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh") pod "network-check-target-shngl" (UID: "7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:46.406463 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.406435    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xft9h\" (UniqueName: \"kubernetes.io/projected/43225ca8-b2b9-42a1-b642-a59a089dd46d-kube-api-access-xft9h\") pod \"ovnkube-node-m2b9r\" (UID: \"43225ca8-b2b9-42a1-b642-a59a089dd46d\") " pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.415659 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.415622    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrsxq\" (UniqueName: \"kubernetes.io/projected/23fc652b-f268-47ef-bb5a-7a8ae8f6d146-kube-api-access-lrsxq\") pod \"multus-additional-cni-plugins-w7lvd\" (UID: \"23fc652b-f268-47ef-bb5a-7a8ae8f6d146\") " pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.416181 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.416100    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxv27\" (UniqueName: \"kubernetes.io/projected/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-kube-api-access-gxv27\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:46.417750 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.417719    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjzck\" (UniqueName: \"kubernetes.io/projected/d510aeee-4e32-4319-a8c6-a47ba4ae9205-kube-api-access-zjzck\") pod \"node-ca-9df2t\" (UID: \"d510aeee-4e32-4319-a8c6-a47ba4ae9205\") " pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.418474 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.418452    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg8fl\" (UniqueName: \"kubernetes.io/projected/d1deda55-7c2d-4b22-97e2-f159a0ddb0e2-kube-api-access-kg8fl\") pod \"aws-ebs-csi-driver-node-4dv6g\" (UID: \"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.418831 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.418806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctnjh\" (UniqueName: \"kubernetes.io/projected/68fa69ca-ef93-41a4-9b62-14e2804b3e26-kube-api-access-ctnjh\") pod \"node-resolver-zfd6l\" (UID: \"68fa69ca-ef93-41a4-9b62-14e2804b3e26\") " pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.472641 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.472603    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vs5zj"
Jul 28 13:59:46.482398 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.482373    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2gpf4"
Jul 28 13:59:46.493552 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.493525    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kphgf"
Jul 28 13:59:46.501445 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.501421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zfd6l"
Jul 28 13:59:46.509624 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.509601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9df2t"
Jul 28 13:59:46.518669 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.518640    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w7lvd"
Jul 28 13:59:46.525549 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.525520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 13:59:46.534351 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.534328    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 13:59:46.540624 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.540601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g"
Jul 28 13:59:46.606113 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:46.606077    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1deda55_7c2d_4b22_97e2_f159a0ddb0e2.slice/crio-2b95c0b13af60e8c61d929bd3cecd37de99a56d9715cdd0791b45cb0d88ead8b WatchSource:0}: Error finding container 2b95c0b13af60e8c61d929bd3cecd37de99a56d9715cdd0791b45cb0d88ead8b: Status 404 returned error can't find the container with id 2b95c0b13af60e8c61d929bd3cecd37de99a56d9715cdd0791b45cb0d88ead8b
Jul 28 13:59:46.607396 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:46.607371    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68fa69ca_ef93_41a4_9b62_14e2804b3e26.slice/crio-6afd11231608bae1d1fee8bbca47570b2a3cc7369e015b726d9e83137d503830 WatchSource:0}: Error finding container 6afd11231608bae1d1fee8bbca47570b2a3cc7369e015b726d9e83137d503830: Status 404 returned error can't find the container with id 6afd11231608bae1d1fee8bbca47570b2a3cc7369e015b726d9e83137d503830
Jul 28 13:59:46.608342 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:46.608276    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23fc652b_f268_47ef_bb5a_7a8ae8f6d146.slice/crio-48b2c47fbc5abf1b409cf51d30e80f127231d84318ba8356de65fbec0555b029 WatchSource:0}: Error finding container 48b2c47fbc5abf1b409cf51d30e80f127231d84318ba8356de65fbec0555b029: Status 404 returned error can't find the container with id 48b2c47fbc5abf1b409cf51d30e80f127231d84318ba8356de65fbec0555b029
Jul 28 13:59:46.610221 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:46.609529    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda75b1cb7_e545_42f1_9f57_6c342726495d.slice/crio-5342288b8facea733597c6d749a097a4cdd8dd5e269f5adeb4a2b661c619fcf7 WatchSource:0}: Error finding container 5342288b8facea733597c6d749a097a4cdd8dd5e269f5adeb4a2b661c619fcf7: Status 404 returned error can't find the container with id 5342288b8facea733597c6d749a097a4cdd8dd5e269f5adeb4a2b661c619fcf7
Jul 28 13:59:46.613179 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:46.610741    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4622aa17_12b9_40ab_9430_e7e2101b8f45.slice/crio-5736c586e2d472517145fa2331a21eb39da63222ee8e18f0ce02c7b3f3aaadb9 WatchSource:0}: Error finding container 5736c586e2d472517145fa2331a21eb39da63222ee8e18f0ce02c7b3f3aaadb9: Status 404 returned error can't find the container with id 5736c586e2d472517145fa2331a21eb39da63222ee8e18f0ce02c7b3f3aaadb9
Jul 28 13:59:46.613451 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:46.613407    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43225ca8_b2b9_42a1_b642_a59a089dd46d.slice/crio-1742ecd6fef81193d83c1593f65c2294c418e19232108fd8494cb075a2566c56 WatchSource:0}: Error finding container 1742ecd6fef81193d83c1593f65c2294c418e19232108fd8494cb075a2566c56: Status 404 returned error can't find the container with id 1742ecd6fef81193d83c1593f65c2294c418e19232108fd8494cb075a2566c56
Jul 28 13:59:46.641122 ip-10-0-143-3 kubenswrapper[2579]: W0728 13:59:46.641089    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd510aeee_4e32_4319_a8c6_a47ba4ae9205.slice/crio-6f6c1b197c4185e18b3cd29f92482ef2e802e9928d269c9ff4111f96c87d0377 WatchSource:0}: Error finding container 6f6c1b197c4185e18b3cd29f92482ef2e802e9928d269c9ff4111f96c87d0377: Status 404 returned error can't find the container with id 6f6c1b197c4185e18b3cd29f92482ef2e802e9928d269c9ff4111f96c87d0377
Jul 28 13:59:46.894443 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.894251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:46.894443 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.894438    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:46.894646 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.894489    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:46.894646 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.894519    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs podName:66bf6b5d-9493-4aaf-aa4f-2f3788bf51be nodeName:}" failed. No retries permitted until 2026-07-28 13:59:47.894489343 +0000 UTC m=+4.021880020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs") pod "network-metrics-daemon-8ltnr" (UID: "66bf6b5d-9493-4aaf-aa4f-2f3788bf51be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:46.894646 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.894588    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:46.894764 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.894655    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret podName:e2f0a8fb-0a74-4e58-8195-47b0ac92551b nodeName:}" failed. No retries permitted until 2026-07-28 13:59:47.894637214 +0000 UTC m=+4.022027899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret") pod "global-pull-secret-syncer-c75dk" (UID: "e2f0a8fb-0a74-4e58-8195-47b0ac92551b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:46.995457 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:46.995415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:46.995637 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.995611    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 13:59:46.995782 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.995645    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 13:59:46.995782 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.995658    2579 projected.go:196] Error preparing data for projected volume kube-api-access-n4knh for pod openshift-network-diagnostics/network-check-target-shngl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:46.995782 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:46.995723    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh podName:7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc nodeName:}" failed. No retries permitted until 2026-07-28 13:59:47.995702738 +0000 UTC m=+4.123093429 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4knh" (UniqueName: "kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh") pod "network-check-target-shngl" (UID: "7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:47.233745 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.233646    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 13:54:46 +0000 UTC" deadline="2028-02-25 12:47:51.132951801 +0000 UTC"
Jul 28 13:59:47.233745 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.233696    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13846h48m3.89926124s"
Jul 28 13:59:47.336618 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.335824    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:47.336618 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:47.335978    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 13:59:47.336618 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.336446    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:47.336618 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:47.336566    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 13:59:47.354934 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.354855    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g" event={"ID":"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2","Type":"ContainerStarted","Data":"2b95c0b13af60e8c61d929bd3cecd37de99a56d9715cdd0791b45cb0d88ead8b"}
Jul 28 13:59:47.358835 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.358804    2579 generic.go:356] "Generic (PLEG): container finished" podID="2dbdcd5940fe0b083a0d80c7c2740683" containerID="92be25d9c79bb2171edfe62bdfa0327996a18a1967749846c853424c90b5c0cf" exitCode=0
Jul 28 13:59:47.358990 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.358969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal" event={"ID":"2dbdcd5940fe0b083a0d80c7c2740683","Type":"ContainerDied","Data":"92be25d9c79bb2171edfe62bdfa0327996a18a1967749846c853424c90b5c0cf"}
Jul 28 13:59:47.360597 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.360552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vs5zj" event={"ID":"846645a8-4f68-42e5-bfcb-abed04caea71","Type":"ContainerStarted","Data":"79f03de73f8248799fda1b311202f9e08f0f71bd44b80d13f303009c3210ef8c"}
Jul 28 13:59:47.364250 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.364222    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9df2t" event={"ID":"d510aeee-4e32-4319-a8c6-a47ba4ae9205","Type":"ContainerStarted","Data":"6f6c1b197c4185e18b3cd29f92482ef2e802e9928d269c9ff4111f96c87d0377"}
Jul 28 13:59:47.366001 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.365973    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jgqhc" event={"ID":"2f4b440f-f03f-4c5e-ad0b-0935b299d316","Type":"ContainerStarted","Data":"3a4884a1ad49813a79d059010941a003e16aee5006123e7fbb606de043810bd8"}
Jul 28 13:59:47.368530 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.368504    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" event={"ID":"43225ca8-b2b9-42a1-b642-a59a089dd46d","Type":"ContainerStarted","Data":"1742ecd6fef81193d83c1593f65c2294c418e19232108fd8494cb075a2566c56"}
Jul 28 13:59:47.371380 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.371353    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2gpf4" event={"ID":"4622aa17-12b9-40ab-9430-e7e2101b8f45","Type":"ContainerStarted","Data":"5736c586e2d472517145fa2331a21eb39da63222ee8e18f0ce02c7b3f3aaadb9"}
Jul 28 13:59:47.373638 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.373612    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kphgf" event={"ID":"a75b1cb7-e545-42f1-9f57-6c342726495d","Type":"ContainerStarted","Data":"5342288b8facea733597c6d749a097a4cdd8dd5e269f5adeb4a2b661c619fcf7"}
Jul 28 13:59:47.377650 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.377568    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w7lvd" event={"ID":"23fc652b-f268-47ef-bb5a-7a8ae8f6d146","Type":"ContainerStarted","Data":"48b2c47fbc5abf1b409cf51d30e80f127231d84318ba8356de65fbec0555b029"}
Jul 28 13:59:47.379560 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.379247    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zfd6l" event={"ID":"68fa69ca-ef93-41a4-9b62-14e2804b3e26","Type":"ContainerStarted","Data":"6afd11231608bae1d1fee8bbca47570b2a3cc7369e015b726d9e83137d503830"}
Jul 28 13:59:47.902239 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.901372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:47.902239 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:47.901443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:47.902239 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:47.901603    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:47.902239 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:47.901677    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret podName:e2f0a8fb-0a74-4e58-8195-47b0ac92551b nodeName:}" failed. No retries permitted until 2026-07-28 13:59:49.901657188 +0000 UTC m=+6.029047878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret") pod "global-pull-secret-syncer-c75dk" (UID: "e2f0a8fb-0a74-4e58-8195-47b0ac92551b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:47.902239 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:47.902132    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:47.902239 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:47.902194    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs podName:66bf6b5d-9493-4aaf-aa4f-2f3788bf51be nodeName:}" failed. No retries permitted until 2026-07-28 13:59:49.902176705 +0000 UTC m=+6.029567391 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs") pod "network-metrics-daemon-8ltnr" (UID: "66bf6b5d-9493-4aaf-aa4f-2f3788bf51be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:48.003318 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:48.002582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:48.003318 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:48.002846    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 13:59:48.003318 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:48.002866    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 13:59:48.003318 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:48.002880    2579 projected.go:196] Error preparing data for projected volume kube-api-access-n4knh for pod openshift-network-diagnostics/network-check-target-shngl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:48.003318 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:48.002960    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh podName:7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc nodeName:}" failed. No retries permitted until 2026-07-28 13:59:50.002931239 +0000 UTC m=+6.130321915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4knh" (UniqueName: "kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh") pod "network-check-target-shngl" (UID: "7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:48.234669 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:48.234546    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 13:54:46 +0000 UTC" deadline="2028-03-27 07:23:49.72447367 +0000 UTC"
Jul 28 13:59:48.234669 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:48.234615    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14585h24m1.489862875s"
Jul 28 13:59:48.339394 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:48.338834    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:48.339394 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:48.338972    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 13:59:48.414506 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:48.414463    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal" event={"ID":"6180e30e1977ed4d8d23fa6e6c33c1ed","Type":"ContainerStarted","Data":"9f114494dcd8d14797b562c9bce4dadc74793ca0e554d4eab18d97d47f3b287e"}
Jul 28 13:59:48.418611 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:48.418541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal" event={"ID":"2dbdcd5940fe0b083a0d80c7c2740683","Type":"ContainerStarted","Data":"ccefacc4a8be9220866f6e168d5b4fcc5990df52577dec78dd2fdd05bd9a6667"}
Jul 28 13:59:48.450808 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:48.450734    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-3.ec2.internal" podStartSLOduration=2.450710782 podStartE2EDuration="2.450710782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:59:48.434791904 +0000 UTC m=+4.562182600" watchObservedRunningTime="2026-07-28 13:59:48.450710782 +0000 UTC m=+4.578101478"
Jul 28 13:59:49.335827 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:49.335790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:49.336365 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:49.335936    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 13:59:49.336457 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:49.336378    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:49.336518 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:49.336494    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 13:59:49.921729 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:49.921681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:49.921918 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:49.921785    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:49.922003 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:49.921926    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:49.922003 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:49.921991    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs podName:66bf6b5d-9493-4aaf-aa4f-2f3788bf51be nodeName:}" failed. No retries permitted until 2026-07-28 13:59:53.921971106 +0000 UTC m=+10.049361795 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs") pod "network-metrics-daemon-8ltnr" (UID: "66bf6b5d-9493-4aaf-aa4f-2f3788bf51be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:49.922163 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:49.922054    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:49.922163 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:49.922112    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret podName:e2f0a8fb-0a74-4e58-8195-47b0ac92551b nodeName:}" failed. No retries permitted until 2026-07-28 13:59:53.922099437 +0000 UTC m=+10.049490124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret") pod "global-pull-secret-syncer-c75dk" (UID: "e2f0a8fb-0a74-4e58-8195-47b0ac92551b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:50.022504 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:50.022441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:50.022708 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:50.022634    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 13:59:50.022708 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:50.022657    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 13:59:50.022708 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:50.022672    2579 projected.go:196] Error preparing data for projected volume kube-api-access-n4knh for pod openshift-network-diagnostics/network-check-target-shngl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:50.022877 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:50.022733    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh podName:7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc nodeName:}" failed. No retries permitted until 2026-07-28 13:59:54.022717635 +0000 UTC m=+10.150108306 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4knh" (UniqueName: "kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh") pod "network-check-target-shngl" (UID: "7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:50.335973 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:50.335937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:50.336487 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:50.336105    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 13:59:51.335469 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:51.335410    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:51.335469 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:51.335436    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:51.335736 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:51.335543    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 13:59:51.335954 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:51.335909    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 13:59:52.336408 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:52.335902    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:52.336408 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:52.336072    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 13:59:53.336142 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:53.336101    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:53.336350 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:53.336248    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 13:59:53.336350 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:53.336322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:53.336470 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:53.336396    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 13:59:53.956621 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:53.956573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:53.956821 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:53.956643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:53.956821 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:53.956785    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:53.956943 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:53.956844    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret podName:e2f0a8fb-0a74-4e58-8195-47b0ac92551b nodeName:}" failed. No retries permitted until 2026-07-28 14:00:01.956828727 +0000 UTC m=+18.084219398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret") pod "global-pull-secret-syncer-c75dk" (UID: "e2f0a8fb-0a74-4e58-8195-47b0ac92551b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 13:59:53.957351 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:53.957202    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:53.957351 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:53.957269    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs podName:66bf6b5d-9493-4aaf-aa4f-2f3788bf51be nodeName:}" failed. No retries permitted until 2026-07-28 14:00:01.957252074 +0000 UTC m=+18.084642758 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs") pod "network-metrics-daemon-8ltnr" (UID: "66bf6b5d-9493-4aaf-aa4f-2f3788bf51be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 13:59:54.058303 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:54.057650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:54.058303 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:54.057872    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 13:59:54.058303 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:54.057894    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 13:59:54.058303 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:54.057908    2579 projected.go:196] Error preparing data for projected volume kube-api-access-n4knh for pod openshift-network-diagnostics/network-check-target-shngl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:54.058303 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:54.057978    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh podName:7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc nodeName:}" failed. No retries permitted until 2026-07-28 14:00:02.057961103 +0000 UTC m=+18.185351773 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4knh" (UniqueName: "kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh") pod "network-check-target-shngl" (UID: "7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 13:59:54.338930 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:54.338893    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:54.339431 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:54.339041    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 13:59:55.335643 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:55.335409    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:55.335643 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:55.335511    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 13:59:55.335643 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:55.335560    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:55.335987 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:55.335710    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 13:59:56.338856 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:56.338826    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:56.339325 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:56.338938    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 13:59:57.336420 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:57.336388    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:57.336583 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:57.336500    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 13:59:57.336583 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:57.336565    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:57.336663 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:57.336638    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 13:59:58.338553 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:58.338524    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 13:59:58.338926 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:58.338640    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 13:59:59.335900 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:59.335869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 13:59:59.336101 ip-10-0-143-3 kubenswrapper[2579]: I0728 13:59:59.335869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 13:59:59.336101 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:59.336004    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 13:59:59.336101 ip-10-0-143-3 kubenswrapper[2579]: E0728 13:59:59.336084    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:00.339211 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:00.339178    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:00.339660 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:00.339308    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:01.335853 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:01.335820    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:01.336131 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:01.335828    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:01.336131 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:01.335942    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:01.336131 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:01.336043    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:02.021247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:02.021211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:02.021639 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:02.021257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:02.021639 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:02.021364    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:00:02.021639 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:02.021395    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:00:02.021639 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:02.021418    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret podName:e2f0a8fb-0a74-4e58-8195-47b0ac92551b nodeName:}" failed. No retries permitted until 2026-07-28 14:00:18.021404938 +0000 UTC m=+34.148795607 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret") pod "global-pull-secret-syncer-c75dk" (UID: "e2f0a8fb-0a74-4e58-8195-47b0ac92551b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:00:02.021639 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:02.021448    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs podName:66bf6b5d-9493-4aaf-aa4f-2f3788bf51be nodeName:}" failed. No retries permitted until 2026-07-28 14:00:18.021434087 +0000 UTC m=+34.148824760 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs") pod "network-metrics-daemon-8ltnr" (UID: "66bf6b5d-9493-4aaf-aa4f-2f3788bf51be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:00:02.122524 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:02.122490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:02.122703 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:02.122683    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:00:02.122769 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:02.122706    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:00:02.122769 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:02.122716    2579 projected.go:196] Error preparing data for projected volume kube-api-access-n4knh for pod openshift-network-diagnostics/network-check-target-shngl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:00:02.122881 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:02.122776    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh podName:7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc nodeName:}" failed. No retries permitted until 2026-07-28 14:00:18.122756848 +0000 UTC m=+34.250147517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4knh" (UniqueName: "kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh") pod "network-check-target-shngl" (UID: "7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:00:02.335453 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:02.335371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:02.335621 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:02.335495    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:03.336457 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:03.336416    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:03.336457 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:03.336443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:03.336963 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:03.336561    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:03.336963 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:03.336666    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:04.336652 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:04.336613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:04.337125 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:04.336723    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:05.336273 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:05.336239    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:05.336457 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:05.336357    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:05.336457 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:05.336413    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:05.336589 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:05.336544    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:06.336417 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.336078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:06.337108 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:06.336526    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:06.463884 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.463852    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g" event={"ID":"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2","Type":"ContainerStarted","Data":"5682ac62f09fd3985066d4561106c039200ef8111048eb7cf4f268997dc6187a"}
Jul 28 14:00:06.465339 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.465312    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vs5zj" event={"ID":"846645a8-4f68-42e5-bfcb-abed04caea71","Type":"ContainerStarted","Data":"d4c554519cf67dfb96a498d450770396cfd9e470cf16e67318c43503ce892786"}
Jul 28 14:00:06.466759 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.466726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9df2t" event={"ID":"d510aeee-4e32-4319-a8c6-a47ba4ae9205","Type":"ContainerStarted","Data":"225dc5c2397acedd77629f8cdb2b2f4af81843724e4ecc5f35ad0ee608eb6cc3"}
Jul 28 14:00:06.468124 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.468102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jgqhc" event={"ID":"2f4b440f-f03f-4c5e-ad0b-0935b299d316","Type":"ContainerStarted","Data":"2408ef9b0b319755790d19ff04ee01a4dee35d2fc16e09023783ea21f2e7f03a"}
Jul 28 14:00:06.470280 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.470258    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" event={"ID":"43225ca8-b2b9-42a1-b642-a59a089dd46d","Type":"ContainerStarted","Data":"cf565804222ad12871aaf2bddf8fd2027c984eb0462910a8cd1d14f1cfcdbfb6"}
Jul 28 14:00:06.470371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.470310    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" event={"ID":"43225ca8-b2b9-42a1-b642-a59a089dd46d","Type":"ContainerStarted","Data":"45686e41eff92b6ee75f73385f3fc0a16e3fd869456fe68ff31a612fdc8b974e"}
Jul 28 14:00:06.470371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.470341    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" event={"ID":"43225ca8-b2b9-42a1-b642-a59a089dd46d","Type":"ContainerStarted","Data":"505e8848274d32f89657e3b727f901452fb26cfb8b08b44e2e6c1c33a5b5660d"}
Jul 28 14:00:06.471774 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.471746    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kphgf" event={"ID":"a75b1cb7-e545-42f1-9f57-6c342726495d","Type":"ContainerStarted","Data":"00eac56e78a306df5a1714501141e2d0f0424e29125afdfa4f06e9cef02bbef4"}
Jul 28 14:00:06.473235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.473208    2579 generic.go:356] "Generic (PLEG): container finished" podID="23fc652b-f268-47ef-bb5a-7a8ae8f6d146" containerID="901e738844bfbad5bd209f0b0dd2018cd16a403637dcc691e350b2ca10bf23da" exitCode=0
Jul 28 14:00:06.473450 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.473285    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w7lvd" event={"ID":"23fc652b-f268-47ef-bb5a-7a8ae8f6d146","Type":"ContainerDied","Data":"901e738844bfbad5bd209f0b0dd2018cd16a403637dcc691e350b2ca10bf23da"}
Jul 28 14:00:06.475090 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.474988    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zfd6l" event={"ID":"68fa69ca-ef93-41a4-9b62-14e2804b3e26","Type":"ContainerStarted","Data":"19ac8afb6f4ed6fd5d31319040e7d9fef271379145787ff3d220aa4582ed7b2a"}
Jul 28 14:00:06.484508 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.484471    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vs5zj" podStartSLOduration=2.008077009 podStartE2EDuration="21.484459287s" totalImagesPullingTime="19.476382278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:06.48388173 +0000 UTC m=+22.611272413" watchObservedRunningTime="2026-07-28 14:00:06.484459287 +0000 UTC m=+22.611849978"
Jul 28 14:00:06.484707 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.484685    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-3.ec2.internal" podStartSLOduration=20.484679723 podStartE2EDuration="20.484679723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:59:48.451969576 +0000 UTC m=+4.579360272" watchObservedRunningTime="2026-07-28 14:00:06.484679723 +0000 UTC m=+22.612070414"
Jul 28 14:00:06.504865 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.504824    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kphgf" podStartSLOduration=2.062411533 podStartE2EDuration="21.504808613s" totalImagesPullingTime="19.44239708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:06.504395834 +0000 UTC m=+22.631786527" watchObservedRunningTime="2026-07-28 14:00:06.504808613 +0000 UTC m=+22.632199305"
Jul 28 14:00:06.520508 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.520468    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zfd6l" podStartSLOduration=2.070534623 podStartE2EDuration="21.520453236s" totalImagesPullingTime="19.449918613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:06.520116295 +0000 UTC m=+22.647506979" watchObservedRunningTime="2026-07-28 14:00:06.520453236 +0000 UTC m=+22.647843927"
Jul 28 14:00:06.562786 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.562740    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jgqhc" podStartSLOduration=2.141190223 podStartE2EDuration="21.562724792s" totalImagesPullingTime="19.421534569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:06.562572131 +0000 UTC m=+22.689962834" watchObservedRunningTime="2026-07-28 14:00:06.562724792 +0000 UTC m=+22.690115482"
Jul 28 14:00:06.578001 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:06.577959    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9df2t" podStartSLOduration=2.166254461 podStartE2EDuration="21.577942846s" totalImagesPullingTime="19.411688385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:06.577906546 +0000 UTC m=+22.705297238" watchObservedRunningTime="2026-07-28 14:00:06.577942846 +0000 UTC m=+22.705333537"
Jul 28 14:00:07.336221 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.336146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:07.336221 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.336174    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:07.336435 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:07.336271    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:07.336435 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:07.336393    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:07.481832 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.481792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" event={"ID":"43225ca8-b2b9-42a1-b642-a59a089dd46d","Type":"ContainerStarted","Data":"f77bc968a16dac3a7147efdafa647d666c8f679ac76f2ba8300b1ea9a555064f"}
Jul 28 14:00:07.482046 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.481898    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" event={"ID":"43225ca8-b2b9-42a1-b642-a59a089dd46d","Type":"ContainerStarted","Data":"db1622b37d339f810cfdc6907520cb3309b92a6121d80f46cbf6c0e32dccf891"}
Jul 28 14:00:07.482046 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.481946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" event={"ID":"43225ca8-b2b9-42a1-b642-a59a089dd46d","Type":"ContainerStarted","Data":"10146f83d6d482cc1863a94a95100e96a11cc8111200e6576d6ea33e11de1abf"}
Jul 28 14:00:07.485953 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.483999    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2gpf4" event={"ID":"4622aa17-12b9-40ab-9430-e7e2101b8f45","Type":"ContainerStarted","Data":"e1f8929f27b9ddc8a8cea86c7bd2b3aa6e69363819de4e737665faf1061d50f2"}
Jul 28 14:00:07.504275 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.504206    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2gpf4" podStartSLOduration=3.084972135 podStartE2EDuration="22.504186933s" totalImagesPullingTime="19.419214798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:07.503832506 +0000 UTC m=+23.631223201" watchObservedRunningTime="2026-07-28 14:00:07.504186933 +0000 UTC m=+23.631577626"
Jul 28 14:00:07.579233 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.579199    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 14:00:07.838110 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.838003    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 14:00:07.838776 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:07.838735    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 14:00:08.243596 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:08.243423    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T14:00:07.579223742Z","UUID":"a1a2c135-e309-49a5-b56c-5349fb98d30e","Handler":null,"Name":"","Endpoint":""}
Jul 28 14:00:08.244857 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:08.244835    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 14:00:08.244941 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:08.244869    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 14:00:08.335519 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:08.335486    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:08.335691 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:08.335602    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:08.489607 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:08.489575    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g" event={"ID":"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2","Type":"ContainerStarted","Data":"7b6685796bc170a1546c8db44bff73b4aceab3e187ec2e54834e5e380d6102d0"}
Jul 28 14:00:09.335977 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:09.335654    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:09.336199 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:09.335717    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:09.336199 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:09.336122    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:09.336302 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:09.336219    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:09.494716 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:09.494639    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" event={"ID":"43225ca8-b2b9-42a1-b642-a59a089dd46d","Type":"ContainerStarted","Data":"bc051a9141031755ce1df033628256b143df1e0c9a22be459d454136b4064e7d"}
Jul 28 14:00:09.498923 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:09.498891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g" event={"ID":"d1deda55-7c2d-4b22-97e2-f159a0ddb0e2","Type":"ContainerStarted","Data":"1fc4af3738caae82b8571826fa711e957067c85d9edcf8bd1d194d3eb00012bf"}
Jul 28 14:00:09.499084 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:09.498953    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 14:00:09.522215 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:09.522161    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dv6g" podStartSLOduration=2.28575811 podStartE2EDuration="24.522144225s" totalImagesPullingTime="22.236386115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:09.521542145 +0000 UTC m=+25.648932848" watchObservedRunningTime="2026-07-28 14:00:09.522144225 +0000 UTC m=+25.649534915"
Jul 28 14:00:10.336336 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:10.336299    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:10.336650 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:10.336435    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:11.335586 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.335397    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:11.336296 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.335450    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:11.336296 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:11.335672    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:11.336296 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:11.335758    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:11.508038 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.507992    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" event={"ID":"43225ca8-b2b9-42a1-b642-a59a089dd46d","Type":"ContainerStarted","Data":"ef406671903d411e287b6d8dccbcdfc4f9285377df693751b1607ea7223e9710"}
Jul 28 14:00:11.508312 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.508277    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 14:00:11.509967 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.509944    2579 generic.go:356] "Generic (PLEG): container finished" podID="23fc652b-f268-47ef-bb5a-7a8ae8f6d146" containerID="fa8b15579877339474468f010c39dd13c9831dd14100be27dd0c3d38e133eb4e" exitCode=0
Jul 28 14:00:11.510090 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.509977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w7lvd" event={"ID":"23fc652b-f268-47ef-bb5a-7a8ae8f6d146","Type":"ContainerDied","Data":"fa8b15579877339474468f010c39dd13c9831dd14100be27dd0c3d38e133eb4e"}
Jul 28 14:00:11.525063 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.524982    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 14:00:11.534775 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.534738    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r" podStartSLOduration=7.043268352 podStartE2EDuration="26.534724598s" totalImagesPullingTime="19.491456246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:11.533823945 +0000 UTC m=+27.661214636" watchObservedRunningTime="2026-07-28 14:00:11.534724598 +0000 UTC m=+27.662115288"
Jul 28 14:00:11.911519 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.911435    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 14:00:11.911681 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.911592    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 14:00:11.912083 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:11.912067    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jgqhc"
Jul 28 14:00:12.336463 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:12.336421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:12.336920 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:12.336554    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:12.512058 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:12.511966    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 14:00:12.512399 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:12.512380    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 14:00:12.528836 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:12.528804    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 14:00:13.335738 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.335705    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:13.335948 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:13.335832    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:13.335948 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.335880    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:13.336086 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:13.335978    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:13.348387 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.348358    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c75dk"]
Jul 28 14:00:13.348902 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.348451    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:13.348902 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:13.348557    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:13.349054 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.348988    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-shngl"]
Jul 28 14:00:13.350356 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.350333    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8ltnr"]
Jul 28 14:00:13.517159 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.517125    2579 generic.go:356] "Generic (PLEG): container finished" podID="23fc652b-f268-47ef-bb5a-7a8ae8f6d146" containerID="0cb724952c04c2897dead82ea0b71c69707ac4bba1f4e675db90a8934bfb4ae1" exitCode=0
Jul 28 14:00:13.517350 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.517229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w7lvd" event={"ID":"23fc652b-f268-47ef-bb5a-7a8ae8f6d146","Type":"ContainerDied","Data":"0cb724952c04c2897dead82ea0b71c69707ac4bba1f4e675db90a8934bfb4ae1"}
Jul 28 14:00:13.517350 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.517286    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:13.517674 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:13.517643    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:13.517674 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:13.517668    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:13.517858 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:13.517771    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:15.336279 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:15.336088    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:15.336663 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:15.336106    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:15.336663 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:15.336360    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:15.336663 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:15.336107    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:15.336663 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:15.336439    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:15.336663 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:15.336534    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:15.524297 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:15.524259    2579 generic.go:356] "Generic (PLEG): container finished" podID="23fc652b-f268-47ef-bb5a-7a8ae8f6d146" containerID="f77d88b0c387183e4ea6c296d3ce7589fdfab2371ebdbda1f963993962b60475" exitCode=0
Jul 28 14:00:15.524459 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:15.524324    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w7lvd" event={"ID":"23fc652b-f268-47ef-bb5a-7a8ae8f6d146","Type":"ContainerDied","Data":"f77d88b0c387183e4ea6c296d3ce7589fdfab2371ebdbda1f963993962b60475"}
Jul 28 14:00:17.336342 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:17.336310    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:17.336948 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:17.336367    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:17.336948 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:17.336310    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:17.336948 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:17.336448    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:17.336948 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:17.336554    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:17.336948 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:17.336624    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:18.035952 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:18.035907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:18.036150 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:18.035967    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:18.036150 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:18.036095    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:00:18.036150 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:18.036135    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:00:18.036302 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:18.036157    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret podName:e2f0a8fb-0a74-4e58-8195-47b0ac92551b nodeName:}" failed. No retries permitted until 2026-07-28 14:00:50.036139127 +0000 UTC m=+66.163529798 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret") pod "global-pull-secret-syncer-c75dk" (UID: "e2f0a8fb-0a74-4e58-8195-47b0ac92551b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:00:18.036302 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:18.036205    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs podName:66bf6b5d-9493-4aaf-aa4f-2f3788bf51be nodeName:}" failed. No retries permitted until 2026-07-28 14:00:50.036185621 +0000 UTC m=+66.163576294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs") pod "network-metrics-daemon-8ltnr" (UID: "66bf6b5d-9493-4aaf-aa4f-2f3788bf51be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:00:18.136488 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:18.136445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:18.136657 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:18.136598    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:00:18.136657 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:18.136613    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:00:18.136657 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:18.136623    2579 projected.go:196] Error preparing data for projected volume kube-api-access-n4knh for pod openshift-network-diagnostics/network-check-target-shngl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:00:18.136780 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:18.136678    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh podName:7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc nodeName:}" failed. No retries permitted until 2026-07-28 14:00:50.136665545 +0000 UTC m=+66.264056215 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4knh" (UniqueName: "kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh") pod "network-check-target-shngl" (UID: "7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:00:19.335482 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.335449    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:19.335482 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.335477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:19.336259 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.335449    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:19.336259 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:19.335589    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shngl" podUID="7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc"
Jul 28 14:00:19.336259 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:19.335735    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltnr" podUID="66bf6b5d-9493-4aaf-aa4f-2f3788bf51be"
Jul 28 14:00:19.336259 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:19.335820    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75dk" podUID="e2f0a8fb-0a74-4e58-8195-47b0ac92551b"
Jul 28 14:00:19.714360 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.714325    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-3.ec2.internal" event="NodeReady"
Jul 28 14:00:19.714520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.714475    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 14:00:19.760883 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.760855    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7657b854fb-p5zb5"]
Jul 28 14:00:19.780608 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.780588    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6"]
Jul 28 14:00:19.780721 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.780702    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.782992 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.782969    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 14:00:19.783322 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.783303    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mljsl\""
Jul 28 14:00:19.783418 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.783359    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 14:00:19.783585 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.783571    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 14:00:19.797205 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.797178    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 14:00:19.807042 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.806980    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"]
Jul 28 14:00:19.807170 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.807112    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6"
Jul 28 14:00:19.809442 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.809422    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.809546 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.809472    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4lnlk\""
Jul 28 14:00:19.809546 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.809421    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.824972 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.824951    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7c97498d97-j9zq7"]
Jul 28 14:00:19.825109 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.825091    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:19.827874 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.827493    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 14:00:19.827874 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.827532    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zzwrj\""
Jul 28 14:00:19.828314 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.828294    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.828502 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.828483    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.828557 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.828532    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 14:00:19.840539 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.840520    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"]
Jul 28 14:00:19.840636 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.840622    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:19.842789 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.842749    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 14:00:19.842966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.842949    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 14:00:19.843039 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.843005    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 14:00:19.843088 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.843047    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j8n4c\""
Jul 28 14:00:19.843088 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.843059    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.843260 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.843247    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.843302 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.843250    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 14:00:19.848004 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.847986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-installation-pull-secrets\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.848094 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.848035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7291bf28-e6bd-4c59-983e-78844bef303e-ca-trust-extracted\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.848094 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.848082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.848163 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.848114    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-trusted-ca\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.848163 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.848152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-registry-certificates\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.848248 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.848187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-bound-sa-token\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.848248 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.848231    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-image-registry-private-configuration\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.848411 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.848253    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws57g\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-kube-api-access-ws57g\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.854619 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.854603    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-crr4w"]
Jul 28 14:00:19.854728 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.854714    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:19.856839 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.856821    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 14:00:19.856932 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.856904    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qlrdg\""
Jul 28 14:00:19.857119 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.857104    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 14:00:19.857119 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.857114    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.857219 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.857138    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.871341 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.871319    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-nhmps"]
Jul 28 14:00:19.871624 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.871544    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:19.874322 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.874122    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 14:00:19.874322 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.874164    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dq5lq\""
Jul 28 14:00:19.874322 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.874177    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.874322 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.874199    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 14:00:19.878031 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.877632    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.880248 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.880225    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 14:00:19.887545 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.887530    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"]
Jul 28 14:00:19.887662 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.887646    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:19.889745 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.889730    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7l84r\""
Jul 28 14:00:19.889852 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.889763    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.889976 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.889963    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 14:00:19.890050 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.890005    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 14:00:19.890134 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.890009    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.895275 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.895255    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 14:00:19.907783 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.907763    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7657b854fb-p5zb5"]
Jul 28 14:00:19.907871 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.907831    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:19.907871 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.907847    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6"]
Jul 28 14:00:19.907871 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.907872    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"]
Jul 28 14:00:19.908030 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.907894    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-khjf6"]
Jul 28 14:00:19.910340 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.910309    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 14:00:19.910340 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.910333    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.910477 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.910343    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.910571 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.910554    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f74xg\""
Jul 28 14:00:19.921965 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.921947    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qqkdd"]
Jul 28 14:00:19.922090 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.922075    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:19.924970 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.924951    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4vfd5\""
Jul 28 14:00:19.925096 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.924978    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 14:00:19.925096 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.924955    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 14:00:19.940129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.940110    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"]
Jul 28 14:00:19.940266 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.940244    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qqkdd"
Jul 28 14:00:19.942395 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.942375    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 14:00:19.942481 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.942375    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wzr9c\""
Jul 28 14:00:19.942481 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.942413    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.942784 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.942769    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.948637 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-registry-certificates\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.948726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85a8fcc2-28d8-4534-908a-c0ea8fdf51ba-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw\" (UID: \"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:19.948726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-bound-sa-token\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.948726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948690    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbrgt\" (UniqueName: \"kubernetes.io/projected/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-kube-api-access-hbrgt\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:19.948894 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c32900e0-0735-40d4-946c-05ec3618add9-snapshots\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:19.948954 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c32900e0-0735-40d4-946c-05ec3618add9-serving-cert\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:19.948954 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948923    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85a8fcc2-28d8-4534-908a-c0ea8fdf51ba-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw\" (UID: \"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:19.948954 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77gfb\" (UniqueName: \"kubernetes.io/projected/85a8fcc2-28d8-4534-908a-c0ea8fdf51ba-kube-api-access-77gfb\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw\" (UID: \"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:19.949118 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c32900e0-0735-40d4-946c-05ec3618add9-tmp\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:19.949118 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948983    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/97a45da1-bd05-4efe-b270-1f7a8bff838e-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:19.949118 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.948999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:19.949118 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-image-registry-private-configuration\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.949296 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws57g\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-kube-api-access-ws57g\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.949296 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-installation-pull-secrets\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.949296 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7291bf28-e6bd-4c59-983e-78844bef303e-ca-trust-extracted\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.949296 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949249    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcrbt\" (UniqueName: \"kubernetes.io/projected/c32900e0-0735-40d4-946c-05ec3618add9-kube-api-access-pcrbt\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:19.949296 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949274    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nkwn\" (UniqueName: \"kubernetes.io/projected/0bfbbd28-0adc-4e9f-99d3-24828951bb8e-kube-api-access-2nkwn\") pod \"volume-data-source-validator-549c6d54cf-68ds6\" (UID: \"0bfbbd28-0adc-4e9f-99d3-24828951bb8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6"
Jul 28 14:00:19.949525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32900e0-0735-40d4-946c-05ec3618add9-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:19.949525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949331    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df52m\" (UniqueName: \"kubernetes.io/projected/97a45da1-bd05-4efe-b270-1f7a8bff838e-kube-api-access-df52m\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:19.949525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949343    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-registry-certificates\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.949525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-default-certificate\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:19.949525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949379    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:19.949525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949409    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-stats-auth\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:19.949525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32900e0-0735-40d4-946c-05ec3618add9-service-ca-bundle\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:19.949525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:19.949525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949525    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.949944 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-trusted-ca\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.949944 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.949647    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7291bf28-e6bd-4c59-983e-78844bef303e-ca-trust-extracted\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.949944 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:19.949711    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 14:00:19.949944 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:19.949731    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7657b854fb-p5zb5: secret "image-registry-tls" not found
Jul 28 14:00:19.949944 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:19.949796    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls podName:7291bf28-e6bd-4c59-983e-78844bef303e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:20.449778337 +0000 UTC m=+36.577169006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls") pod "image-registry-7657b854fb-p5zb5" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e") : secret "image-registry-tls" not found
Jul 28 14:00:19.950442 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.950422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-trusted-ca\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.953273 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.953256    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-installation-pull-secrets\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.953273 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.953268    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-image-registry-private-configuration\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.962417 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.962398    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-bound-sa-token\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.962532 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.962482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws57g\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-kube-api-access-ws57g\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:19.976541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.976519    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"]
Jul 28 14:00:19.976666 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.976550    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n"]
Jul 28 14:00:19.976711 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.976664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:19.979382 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.979360    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.979501 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.979414    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 14:00:19.979501 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.979424    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.979501 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.979442    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hn82n\""
Jul 28 14:00:19.979501 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.979421    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 14:00:19.994466 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.994441    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"]
Jul 28 14:00:19.994569 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.994489    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-nhmps"]
Jul 28 14:00:19.994569 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.994509    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-crr4w"]
Jul 28 14:00:19.994569 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.994531    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c97498d97-j9zq7"]
Jul 28 14:00:19.994569 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.994539    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n"
Jul 28 14:00:19.994569 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.994550    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"]
Jul 28 14:00:19.994569 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.994567    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qqkdd"]
Jul 28 14:00:19.994789 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.994586    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-khjf6"]
Jul 28 14:00:19.994789 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.994614    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n"]
Jul 28 14:00:19.996873 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.996853    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 14:00:19.996968 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.996886    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 14:00:19.997060 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:19.996965    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7nlhp\""
Jul 28 14:00:20.050876 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.050685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcrbt\" (UniqueName: \"kubernetes.io/projected/c32900e0-0735-40d4-946c-05ec3618add9-kube-api-access-pcrbt\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.050976 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.050894    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/943d836b-9abc-4857-9eb8-ea5ae2a76793-cert\") pod \"ingress-canary-qqkdd\" (UID: \"943d836b-9abc-4857-9eb8-ea5ae2a76793\") " pod="openshift-ingress-canary/ingress-canary-qqkdd"
Jul 28 14:00:20.050976 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.050918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32900e0-0735-40d4-946c-05ec3618add9-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.050976 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.050935    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-stats-auth\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.050976 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.050950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4915355-1200-44ec-a2ef-2a8ace1c6348-config\") pod \"service-ca-operator-5d6c75f84f-72w7b\" (UID: \"b4915355-1200-44ec-a2ef-2a8ace1c6348\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.050976 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.050968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.051253 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.050997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-config\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.051253 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77gfb\" (UniqueName: \"kubernetes.io/projected/85a8fcc2-28d8-4534-908a-c0ea8fdf51ba-kube-api-access-77gfb\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw\" (UID: \"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:20.051253 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051137    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c32900e0-0735-40d4-946c-05ec3618add9-tmp\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.051253 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-tmp-dir\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.051454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051251    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:20.051454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051296    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85a8fcc2-28d8-4534-908a-c0ea8fdf51ba-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw\" (UID: \"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:20.051454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csg8s\" (UniqueName: \"kubernetes.io/projected/5833ae33-c691-4b49-a25b-d221280278d8-kube-api-access-csg8s\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:20.051454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.051454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051382    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9qkg\" (UniqueName: \"kubernetes.io/projected/b4915355-1200-44ec-a2ef-2a8ace1c6348-kube-api-access-s9qkg\") pod \"service-ca-operator-5d6c75f84f-72w7b\" (UID: \"b4915355-1200-44ec-a2ef-2a8ace1c6348\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.051686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051456    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df52m\" (UniqueName: \"kubernetes.io/projected/97a45da1-bd05-4efe-b270-1f7a8bff838e-kube-api-access-df52m\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:20.051686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051467    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c32900e0-0735-40d4-946c-05ec3618add9-tmp\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.051686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rbj8\" (UniqueName: \"kubernetes.io/projected/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-kube-api-access-9rbj8\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.051686 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.051532    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:20.551507205 +0000 UTC m=+36.678897893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : configmap references non-existent config key: service-ca.crt
Jul 28 14:00:20.051686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051562    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32900e0-0735-40d4-946c-05ec3618add9-service-ca-bundle\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.051686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-trusted-ca\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.051686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.051686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051655    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvx6w\" (UniqueName: \"kubernetes.io/projected/943d836b-9abc-4857-9eb8-ea5ae2a76793-kube-api-access-mvx6w\") pod \"ingress-canary-qqkdd\" (UID: \"943d836b-9abc-4857-9eb8-ea5ae2a76793\") " pod="openshift-ingress-canary/ingress-canary-qqkdd"
Jul 28 14:00:20.051686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051679    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-serving-cert\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85a8fcc2-28d8-4534-908a-c0ea8fdf51ba-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw\" (UID: \"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.051748    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-config-volume\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbrgt\" (UniqueName: \"kubernetes.io/projected/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-kube-api-access-hbrgt\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.051799    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:20.551782467 +0000 UTC m=+36.679173144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : secret "router-metrics-certs-default" not found
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c32900e0-0735-40d4-946c-05ec3618add9-snapshots\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85a8fcc2-28d8-4534-908a-c0ea8fdf51ba-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw\" (UID: \"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c32900e0-0735-40d4-946c-05ec3618add9-serving-cert\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/97a45da1-bd05-4efe-b270-1f7a8bff838e-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051927    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.051954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4915355-1200-44ec-a2ef-2a8ace1c6348-serving-cert\") pod \"service-ca-operator-5d6c75f84f-72w7b\" (UID: \"b4915355-1200-44ec-a2ef-2a8ace1c6348\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.052047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2vh6\" (UniqueName: \"kubernetes.io/projected/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-kube-api-access-w2vh6\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.052063    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32900e0-0735-40d4-946c-05ec3618add9-service-ca-bundle\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.052076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-default-certificate\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.052195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.052135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nkwn\" (UniqueName: \"kubernetes.io/projected/0bfbbd28-0adc-4e9f-99d3-24828951bb8e-kube-api-access-2nkwn\") pod \"volume-data-source-validator-549c6d54cf-68ds6\" (UID: \"0bfbbd28-0adc-4e9f-99d3-24828951bb8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6"
Jul 28 14:00:20.053174 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.052417    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:20.053174 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.052470    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls podName:97a45da1-bd05-4efe-b270-1f7a8bff838e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:20.552453632 +0000 UTC m=+36.679844303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-2rhml" (UID: "97a45da1-bd05-4efe-b270-1f7a8bff838e") : secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:20.053174 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.052508    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c32900e0-0735-40d4-946c-05ec3618add9-snapshots\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.053174 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.052643    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/97a45da1-bd05-4efe-b270-1f7a8bff838e-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:20.053809 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.053786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-stats-auth\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.054183 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.054167    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85a8fcc2-28d8-4534-908a-c0ea8fdf51ba-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw\" (UID: \"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:20.054881 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.054856    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c32900e0-0735-40d4-946c-05ec3618add9-serving-cert\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.054968 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.054955    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-default-certificate\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.062782 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.062755    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df52m\" (UniqueName: \"kubernetes.io/projected/97a45da1-bd05-4efe-b270-1f7a8bff838e-kube-api-access-df52m\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:20.062956 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.062934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32900e0-0735-40d4-946c-05ec3618add9-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.063355 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.063336    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77gfb\" (UniqueName: \"kubernetes.io/projected/85a8fcc2-28d8-4534-908a-c0ea8fdf51ba-kube-api-access-77gfb\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw\" (UID: \"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:20.063552 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.063532    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcrbt\" (UniqueName: \"kubernetes.io/projected/c32900e0-0735-40d4-946c-05ec3618add9-kube-api-access-pcrbt\") pod \"insights-operator-5885fdb759-crr4w\" (UID: \"c32900e0-0735-40d4-946c-05ec3618add9\") " pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.064067 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.064052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbrgt\" (UniqueName: \"kubernetes.io/projected/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-kube-api-access-hbrgt\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.064151 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.064137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nkwn\" (UniqueName: \"kubernetes.io/projected/0bfbbd28-0adc-4e9f-99d3-24828951bb8e-kube-api-access-2nkwn\") pod \"volume-data-source-validator-549c6d54cf-68ds6\" (UID: \"0bfbbd28-0adc-4e9f-99d3-24828951bb8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6"
Jul 28 14:00:20.120066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.119986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6"
Jul 28 14:00:20.153394 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbj8\" (UniqueName: \"kubernetes.io/projected/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-kube-api-access-9rbj8\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.153489 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153402    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-trusted-ca\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.153489 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvx6w\" (UniqueName: \"kubernetes.io/projected/943d836b-9abc-4857-9eb8-ea5ae2a76793-kube-api-access-mvx6w\") pod \"ingress-canary-qqkdd\" (UID: \"943d836b-9abc-4857-9eb8-ea5ae2a76793\") " pod="openshift-ingress-canary/ingress-canary-qqkdd"
Jul 28 14:00:20.153489 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-serving-cert\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.153698 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srj7f\" (UniqueName: \"kubernetes.io/projected/d045014b-e31a-4564-9698-fb0e41bdbecb-kube-api-access-srj7f\") pod \"network-check-source-7d88cb55d8-wqs5n\" (UID: \"d045014b-e31a-4564-9698-fb0e41bdbecb\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n"
Jul 28 14:00:20.153698 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-config-volume\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.153698 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4915355-1200-44ec-a2ef-2a8ace1c6348-serving-cert\") pod \"service-ca-operator-5d6c75f84f-72w7b\" (UID: \"b4915355-1200-44ec-a2ef-2a8ace1c6348\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.153698 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153652    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vh6\" (UniqueName: \"kubernetes.io/projected/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-kube-api-access-w2vh6\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.153899 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153716    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/943d836b-9abc-4857-9eb8-ea5ae2a76793-cert\") pod \"ingress-canary-qqkdd\" (UID: \"943d836b-9abc-4857-9eb8-ea5ae2a76793\") " pod="openshift-ingress-canary/ingress-canary-qqkdd"
Jul 28 14:00:20.153899 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153755    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4915355-1200-44ec-a2ef-2a8ace1c6348-config\") pod \"service-ca-operator-5d6c75f84f-72w7b\" (UID: \"b4915355-1200-44ec-a2ef-2a8ace1c6348\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.153899 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.153899 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-config\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.153899 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-tmp-dir\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.153899 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:20.154192 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153919    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csg8s\" (UniqueName: \"kubernetes.io/projected/5833ae33-c691-4b49-a25b-d221280278d8-kube-api-access-csg8s\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:20.154192 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.153958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9qkg\" (UniqueName: \"kubernetes.io/projected/b4915355-1200-44ec-a2ef-2a8ace1c6348-kube-api-access-s9qkg\") pod \"service-ca-operator-5d6c75f84f-72w7b\" (UID: \"b4915355-1200-44ec-a2ef-2a8ace1c6348\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.154192 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.154071    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:00:20.154192 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.154137    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls podName:5833ae33-c691-4b49-a25b-d221280278d8 nodeName:}" failed. No retries permitted until 2026-07-28 14:00:20.654115754 +0000 UTC m=+36.781506423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls") pod "cluster-samples-operator-6748489656-fkksj" (UID: "5833ae33-c691-4b49-a25b-d221280278d8") : secret "samples-operator-tls" not found
Jul 28 14:00:20.154192 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.154168    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-config-volume\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.154429 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.154306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-trusted-ca\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.154696 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.154559    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:00:20.154696 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.154635    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls podName:1e4ec977-d8d3-4bd6-9bb0-eb5df490604c nodeName:}" failed. No retries permitted until 2026-07-28 14:00:20.654618308 +0000 UTC m=+36.782008979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls") pod "dns-default-khjf6" (UID: "1e4ec977-d8d3-4bd6-9bb0-eb5df490604c") : secret "dns-default-metrics-tls" not found
Jul 28 14:00:20.155077 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.154852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4915355-1200-44ec-a2ef-2a8ace1c6348-config\") pod \"service-ca-operator-5d6c75f84f-72w7b\" (UID: \"b4915355-1200-44ec-a2ef-2a8ace1c6348\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.155077 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.154882    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-tmp-dir\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.156549 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.156528    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4915355-1200-44ec-a2ef-2a8ace1c6348-serving-cert\") pod \"service-ca-operator-5d6c75f84f-72w7b\" (UID: \"b4915355-1200-44ec-a2ef-2a8ace1c6348\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.156950 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.156934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/943d836b-9abc-4857-9eb8-ea5ae2a76793-cert\") pod \"ingress-canary-qqkdd\" (UID: \"943d836b-9abc-4857-9eb8-ea5ae2a76793\") " pod="openshift-ingress-canary/ingress-canary-qqkdd"
Jul 28 14:00:20.156995 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.156934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-serving-cert\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.161778 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.161753    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-config\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.163376 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.163331    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvx6w\" (UniqueName: \"kubernetes.io/projected/943d836b-9abc-4857-9eb8-ea5ae2a76793-kube-api-access-mvx6w\") pod \"ingress-canary-qqkdd\" (UID: \"943d836b-9abc-4857-9eb8-ea5ae2a76793\") " pod="openshift-ingress-canary/ingress-canary-qqkdd"
Jul 28 14:00:20.163474 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.163421    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2vh6\" (UniqueName: \"kubernetes.io/projected/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-kube-api-access-w2vh6\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.164270 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.164244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rbj8\" (UniqueName: \"kubernetes.io/projected/70a294fd-3ee6-40f5-b772-5b4d8dca14fd-kube-api-access-9rbj8\") pod \"console-operator-584dcb6b54-nhmps\" (UID: \"70a294fd-3ee6-40f5-b772-5b4d8dca14fd\") " pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.164523 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.164509    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csg8s\" (UniqueName: \"kubernetes.io/projected/5833ae33-c691-4b49-a25b-d221280278d8-kube-api-access-csg8s\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:20.164847 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.164833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9qkg\" (UniqueName: \"kubernetes.io/projected/b4915355-1200-44ec-a2ef-2a8ace1c6348-kube-api-access-s9qkg\") pod \"service-ca-operator-5d6c75f84f-72w7b\" (UID: \"b4915355-1200-44ec-a2ef-2a8ace1c6348\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.176052 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.175837    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"
Jul 28 14:00:20.182930 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.182903    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-crr4w"
Jul 28 14:00:20.196521 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.196273    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:20.250701 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.248594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qqkdd"
Jul 28 14:00:20.256120 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.255577    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srj7f\" (UniqueName: \"kubernetes.io/projected/d045014b-e31a-4564-9698-fb0e41bdbecb-kube-api-access-srj7f\") pod \"network-check-source-7d88cb55d8-wqs5n\" (UID: \"d045014b-e31a-4564-9698-fb0e41bdbecb\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n"
Jul 28 14:00:20.268214 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.268158    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srj7f\" (UniqueName: \"kubernetes.io/projected/d045014b-e31a-4564-9698-fb0e41bdbecb-kube-api-access-srj7f\") pod \"network-check-source-7d88cb55d8-wqs5n\" (UID: \"d045014b-e31a-4564-9698-fb0e41bdbecb\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n"
Jul 28 14:00:20.291464 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.291242    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"
Jul 28 14:00:20.304098 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.303763    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n"
Jul 28 14:00:20.321677 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.320617    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6"]
Jul 28 14:00:20.330703 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:20.330641    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bfbbd28_0adc_4e9f_99d3_24828951bb8e.slice/crio-2bcb8b01c5d5165adb403a91cc16d4db8f9214299dd71d3b1738d27be82c80c5 WatchSource:0}: Error finding container 2bcb8b01c5d5165adb403a91cc16d4db8f9214299dd71d3b1738d27be82c80c5: Status 404 returned error can't find the container with id 2bcb8b01c5d5165adb403a91cc16d4db8f9214299dd71d3b1738d27be82c80c5
Jul 28 14:00:20.350800 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.350725    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw"]
Jul 28 14:00:20.356579 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:20.356148    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a8fcc2_28d8_4534_908a_c0ea8fdf51ba.slice/crio-2a40a2e84d38d0611e9e9cafd890970b310e023880e2abab7cfb3f676852fecf WatchSource:0}: Error finding container 2a40a2e84d38d0611e9e9cafd890970b310e023880e2abab7cfb3f676852fecf: Status 404 returned error can't find the container with id 2a40a2e84d38d0611e9e9cafd890970b310e023880e2abab7cfb3f676852fecf
Jul 28 14:00:20.367921 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.367889    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-crr4w"]
Jul 28 14:00:20.376520 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:20.374411    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc32900e0_0735_40d4_946c_05ec3618add9.slice/crio-5c1b8add71f6d4553752fb9b98512f8c51c87569641d4b62ab8cd2115e045fbc WatchSource:0}: Error finding container 5c1b8add71f6d4553752fb9b98512f8c51c87569641d4b62ab8cd2115e045fbc: Status 404 returned error can't find the container with id 5c1b8add71f6d4553752fb9b98512f8c51c87569641d4b62ab8cd2115e045fbc
Jul 28 14:00:20.391495 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.391008    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-nhmps"]
Jul 28 14:00:20.415897 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:20.415851    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a294fd_3ee6_40f5_b772_5b4d8dca14fd.slice/crio-181ae88f8457ed8a85303bfc6106ab36c483fb924d8a9ab42dffca0b9398bab7 WatchSource:0}: Error finding container 181ae88f8457ed8a85303bfc6106ab36c483fb924d8a9ab42dffca0b9398bab7: Status 404 returned error can't find the container with id 181ae88f8457ed8a85303bfc6106ab36c483fb924d8a9ab42dffca0b9398bab7
Jul 28 14:00:20.454748 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.454712    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qqkdd"]
Jul 28 14:00:20.457632 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.457604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:20.457770 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.457750    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 14:00:20.457814 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.457775    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7657b854fb-p5zb5: secret "image-registry-tls" not found
Jul 28 14:00:20.457858 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.457846    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls podName:7291bf28-e6bd-4c59-983e-78844bef303e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:21.457825213 +0000 UTC m=+37.585215902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls") pod "image-registry-7657b854fb-p5zb5" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e") : secret "image-registry-tls" not found
Jul 28 14:00:20.458582 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:20.458558    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943d836b_9abc_4857_9eb8_ea5ae2a76793.slice/crio-f61f27ee2f806a5ee3d00a2e2b967ed2644e595d8bce8240c5be29fa550ad3b9 WatchSource:0}: Error finding container f61f27ee2f806a5ee3d00a2e2b967ed2644e595d8bce8240c5be29fa550ad3b9: Status 404 returned error can't find the container with id f61f27ee2f806a5ee3d00a2e2b967ed2644e595d8bce8240c5be29fa550ad3b9
Jul 28 14:00:20.481486 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.481465    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b"]
Jul 28 14:00:20.485693 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:20.485653    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4915355_1200_44ec_a2ef_2a8ace1c6348.slice/crio-a0852b83f24d40776895a21a43d33f888a911be6f8348ce8699260030f65dec3 WatchSource:0}: Error finding container a0852b83f24d40776895a21a43d33f888a911be6f8348ce8699260030f65dec3: Status 404 returned error can't find the container with id a0852b83f24d40776895a21a43d33f888a911be6f8348ce8699260030f65dec3
Jul 28 14:00:20.486121 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.486097    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n"]
Jul 28 14:00:20.488467 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:20.488447    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd045014b_e31a_4564_9698_fb0e41bdbecb.slice/crio-ba9d44a28c899477b115343b7daa7370f4125ae41e40740c878950a34f64bffe WatchSource:0}: Error finding container ba9d44a28c899477b115343b7daa7370f4125ae41e40740c878950a34f64bffe: Status 404 returned error can't find the container with id ba9d44a28c899477b115343b7daa7370f4125ae41e40740c878950a34f64bffe
Jul 28 14:00:20.540868 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.540836    2579 generic.go:356] "Generic (PLEG): container finished" podID="23fc652b-f268-47ef-bb5a-7a8ae8f6d146" containerID="156e46f6c0ee3086ed12cf0544f16ad304e94313d6b1cfa3b47b1a783b9ac8f0" exitCode=0
Jul 28 14:00:20.541004 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.540916    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w7lvd" event={"ID":"23fc652b-f268-47ef-bb5a-7a8ae8f6d146","Type":"ContainerDied","Data":"156e46f6c0ee3086ed12cf0544f16ad304e94313d6b1cfa3b47b1a783b9ac8f0"}
Jul 28 14:00:20.542128 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.542094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-crr4w" event={"ID":"c32900e0-0735-40d4-946c-05ec3618add9","Type":"ContainerStarted","Data":"5c1b8add71f6d4553752fb9b98512f8c51c87569641d4b62ab8cd2115e045fbc"}
Jul 28 14:00:20.543068 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.543043    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6" event={"ID":"0bfbbd28-0adc-4e9f-99d3-24828951bb8e","Type":"ContainerStarted","Data":"2bcb8b01c5d5165adb403a91cc16d4db8f9214299dd71d3b1738d27be82c80c5"}
Jul 28 14:00:20.544230 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.544201    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" event={"ID":"70a294fd-3ee6-40f5-b772-5b4d8dca14fd","Type":"ContainerStarted","Data":"181ae88f8457ed8a85303bfc6106ab36c483fb924d8a9ab42dffca0b9398bab7"}
Jul 28 14:00:20.545326 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.545304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n" event={"ID":"d045014b-e31a-4564-9698-fb0e41bdbecb","Type":"ContainerStarted","Data":"ba9d44a28c899477b115343b7daa7370f4125ae41e40740c878950a34f64bffe"}
Jul 28 14:00:20.546412 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.546391    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b" event={"ID":"b4915355-1200-44ec-a2ef-2a8ace1c6348","Type":"ContainerStarted","Data":"a0852b83f24d40776895a21a43d33f888a911be6f8348ce8699260030f65dec3"}
Jul 28 14:00:20.547445 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.547426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw" event={"ID":"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba","Type":"ContainerStarted","Data":"2a40a2e84d38d0611e9e9cafd890970b310e023880e2abab7cfb3f676852fecf"}
Jul 28 14:00:20.548434 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.548415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qqkdd" event={"ID":"943d836b-9abc-4857-9eb8-ea5ae2a76793","Type":"ContainerStarted","Data":"f61f27ee2f806a5ee3d00a2e2b967ed2644e595d8bce8240c5be29fa550ad3b9"}
Jul 28 14:00:20.558368 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.558342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.558442 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.558396    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:20.558491 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.558450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:20.558546 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.558526    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:21.558503309 +0000 UTC m=+37.685894001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : configmap references non-existent config key: service-ca.crt
Jul 28 14:00:20.558601 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.558565    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:20.558601 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.558590    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 14:00:20.558661 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.558607    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls podName:97a45da1-bd05-4efe-b270-1f7a8bff838e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:21.558591758 +0000 UTC m=+37.685982427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-2rhml" (UID: "97a45da1-bd05-4efe-b270-1f7a8bff838e") : secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:20.558661 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.558637    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:21.558627089 +0000 UTC m=+37.686017759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : secret "router-metrics-certs-default" not found
Jul 28 14:00:20.659555 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.659527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:20.659678 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:20.659580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:20.659742 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.659700    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:00:20.659742 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.659710    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:00:20.659824 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.659770    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls podName:1e4ec977-d8d3-4bd6-9bb0-eb5df490604c nodeName:}" failed. No retries permitted until 2026-07-28 14:00:21.659750484 +0000 UTC m=+37.787141174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls") pod "dns-default-khjf6" (UID: "1e4ec977-d8d3-4bd6-9bb0-eb5df490604c") : secret "dns-default-metrics-tls" not found
Jul 28 14:00:20.659824 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:20.659792    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls podName:5833ae33-c691-4b49-a25b-d221280278d8 nodeName:}" failed. No retries permitted until 2026-07-28 14:00:21.659779878 +0000 UTC m=+37.787170546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls") pod "cluster-samples-operator-6748489656-fkksj" (UID: "5833ae33-c691-4b49-a25b-d221280278d8") : secret "samples-operator-tls" not found
Jul 28 14:00:21.336419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.336169    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:21.336871 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.336855    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:21.337325 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.337306    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:21.339798 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.339093    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-96fnv\""
Jul 28 14:00:21.339798 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.339776    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vgxlf\""
Jul 28 14:00:21.339798 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.339791    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 14:00:21.339986 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.339836    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 14:00:21.466740 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.466702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:21.467223 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.467112    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 14:00:21.467223 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.467130    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7657b854fb-p5zb5: secret "image-registry-tls" not found
Jul 28 14:00:21.467223 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.467193    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls podName:7291bf28-e6bd-4c59-983e-78844bef303e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:23.46717373 +0000 UTC m=+39.594564420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls") pod "image-registry-7657b854fb-p5zb5" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e") : secret "image-registry-tls" not found
Jul 28 14:00:21.560747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.560710    2579 generic.go:356] "Generic (PLEG): container finished" podID="23fc652b-f268-47ef-bb5a-7a8ae8f6d146" containerID="0473244cc67727ed04e1096533af1f1d8509dc08e73ba4eb86c7a28e424d1e75" exitCode=0
Jul 28 14:00:21.560906 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.560802    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w7lvd" event={"ID":"23fc652b-f268-47ef-bb5a-7a8ae8f6d146","Type":"ContainerDied","Data":"0473244cc67727ed04e1096533af1f1d8509dc08e73ba4eb86c7a28e424d1e75"}
Jul 28 14:00:21.567363 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.567334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:21.567470 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.567408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:21.567531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.567467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:21.567593 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.567554    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:23.567531236 +0000 UTC m=+39.694921917 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : configmap references non-existent config key: service-ca.crt
Jul 28 14:00:21.567593 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.567589    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:21.567712 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.567640    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls podName:97a45da1-bd05-4efe-b270-1f7a8bff838e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:23.567624152 +0000 UTC m=+39.695014823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-2rhml" (UID: "97a45da1-bd05-4efe-b270-1f7a8bff838e") : secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:21.567712 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.567708    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 14:00:21.567812 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.567743    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:23.567731171 +0000 UTC m=+39.695121844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : secret "router-metrics-certs-default" not found
Jul 28 14:00:21.669320 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.669293    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:21.669614 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:21.669336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:21.669614 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.669481    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:00:21.669614 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.669529    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls podName:5833ae33-c691-4b49-a25b-d221280278d8 nodeName:}" failed. No retries permitted until 2026-07-28 14:00:23.669516463 +0000 UTC m=+39.796907132 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls") pod "cluster-samples-operator-6748489656-fkksj" (UID: "5833ae33-c691-4b49-a25b-d221280278d8") : secret "samples-operator-tls" not found
Jul 28 14:00:21.669850 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.669833    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:00:21.669906 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:21.669870    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls podName:1e4ec977-d8d3-4bd6-9bb0-eb5df490604c nodeName:}" failed. No retries permitted until 2026-07-28 14:00:23.669860169 +0000 UTC m=+39.797250838 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls") pod "dns-default-khjf6" (UID: "1e4ec977-d8d3-4bd6-9bb0-eb5df490604c") : secret "dns-default-metrics-tls" not found
Jul 28 14:00:22.583373 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:22.583338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w7lvd" event={"ID":"23fc652b-f268-47ef-bb5a-7a8ae8f6d146","Type":"ContainerStarted","Data":"4f5d412213276531a52f4f632a47a3b661f69457cacc0577b2beb585d609bd0f"}
Jul 28 14:00:22.609557 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:22.609481    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w7lvd" podStartSLOduration=6.900026598 podStartE2EDuration="37.609463454s" totalImagesPullingTime="30.709436856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:22.608842877 +0000 UTC m=+38.736233625" watchObservedRunningTime="2026-07-28 14:00:22.609463454 +0000 UTC m=+38.736854136"
Jul 28 14:00:23.488721 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:23.488683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:23.488900 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.488826    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 14:00:23.488900 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.488843    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7657b854fb-p5zb5: secret "image-registry-tls" not found
Jul 28 14:00:23.489046 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.488908    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls podName:7291bf28-e6bd-4c59-983e-78844bef303e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:27.488889724 +0000 UTC m=+43.616280393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls") pod "image-registry-7657b854fb-p5zb5" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e") : secret "image-registry-tls" not found
Jul 28 14:00:23.589115 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:23.589064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6" event={"ID":"0bfbbd28-0adc-4e9f-99d3-24828951bb8e","Type":"ContainerStarted","Data":"a006f6b302cd0df09c1883a5805c80b2fa158f21efc7a5f40062ac553372d3c5"}
Jul 28 14:00:23.589677 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:23.589166    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:23.589677 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.589459    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:27.589422443 +0000 UTC m=+43.716813132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : configmap references non-existent config key: service-ca.crt
Jul 28 14:00:23.589677 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:23.589495    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:23.589677 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:23.589580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:23.589867 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.589751    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 14:00:23.589867 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.589816    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:27.589793055 +0000 UTC m=+43.717183783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : secret "router-metrics-certs-default" not found
Jul 28 14:00:23.589928 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.589875    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:23.589928 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.589918    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls podName:97a45da1-bd05-4efe-b270-1f7a8bff838e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:27.589906519 +0000 UTC m=+43.717297187 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-2rhml" (UID: "97a45da1-bd05-4efe-b270-1f7a8bff838e") : secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:23.617965 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:23.614412    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-68ds6" podStartSLOduration=15.516511238 podStartE2EDuration="17.614391827s" totalImagesPullingTime="2.097880589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:23.614132155 +0000 UTC m=+39.741522847" watchObservedRunningTime="2026-07-28 14:00:23.614391827 +0000 UTC m=+39.741782519"
Jul 28 14:00:23.690797 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:23.690756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:23.690969 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:23.690809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:23.691067 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.690975    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:00:23.691067 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.691058    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls podName:5833ae33-c691-4b49-a25b-d221280278d8 nodeName:}" failed. No retries permitted until 2026-07-28 14:00:27.691039499 +0000 UTC m=+43.818430191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls") pod "cluster-samples-operator-6748489656-fkksj" (UID: "5833ae33-c691-4b49-a25b-d221280278d8") : secret "samples-operator-tls" not found
Jul 28 14:00:23.691243 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.691218    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:00:23.691356 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:23.691266    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls podName:1e4ec977-d8d3-4bd6-9bb0-eb5df490604c nodeName:}" failed. No retries permitted until 2026-07-28 14:00:27.691253896 +0000 UTC m=+43.818644579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls") pod "dns-default-khjf6" (UID: "1e4ec977-d8d3-4bd6-9bb0-eb5df490604c") : secret "dns-default-metrics-tls" not found
Jul 28 14:00:27.525556 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:27.525507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:27.525991 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.525656    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 14:00:27.525991 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.525672    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7657b854fb-p5zb5: secret "image-registry-tls" not found
Jul 28 14:00:27.525991 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.525729    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls podName:7291bf28-e6bd-4c59-983e-78844bef303e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:35.52571318 +0000 UTC m=+51.653103851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls") pod "image-registry-7657b854fb-p5zb5" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e") : secret "image-registry-tls" not found
Jul 28 14:00:27.626413 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:27.626370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:27.626598 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:27.626446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:27.626598 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.626564    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 14:00:27.626710 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:27.626613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:27.626710 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.626627    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:27.626710 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.626649    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:35.6266232 +0000 UTC m=+51.754013884 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : secret "router-metrics-certs-default" not found
Jul 28 14:00:27.626710 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.626689    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls podName:97a45da1-bd05-4efe-b270-1f7a8bff838e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:35.626671908 +0000 UTC m=+51.754062577 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-2rhml" (UID: "97a45da1-bd05-4efe-b270-1f7a8bff838e") : secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:27.626710 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.626711    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:35.626700942 +0000 UTC m=+51.754091612 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : configmap references non-existent config key: service-ca.crt
Jul 28 14:00:27.727362 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:27.727319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:27.727541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:27.727398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:27.727541 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.727460    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:00:27.727649 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.727548    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls podName:1e4ec977-d8d3-4bd6-9bb0-eb5df490604c nodeName:}" failed. No retries permitted until 2026-07-28 14:00:35.727525788 +0000 UTC m=+51.854916460 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls") pod "dns-default-khjf6" (UID: "1e4ec977-d8d3-4bd6-9bb0-eb5df490604c") : secret "dns-default-metrics-tls" not found
Jul 28 14:00:27.727649 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.727590    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:00:27.727649 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:27.727646    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls podName:5833ae33-c691-4b49-a25b-d221280278d8 nodeName:}" failed. No retries permitted until 2026-07-28 14:00:35.727630154 +0000 UTC m=+51.855020845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls") pod "cluster-samples-operator-6748489656-fkksj" (UID: "5833ae33-c691-4b49-a25b-d221280278d8") : secret "samples-operator-tls" not found
Jul 28 14:00:29.607638 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.607603    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/0.log"
Jul 28 14:00:29.608167 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.607652    2579 generic.go:356] "Generic (PLEG): container finished" podID="70a294fd-3ee6-40f5-b772-5b4d8dca14fd" containerID="c7198a87ba641d9ff107b3829f6dfc807845a19c90034afaf3d7b20e32d1b77d" exitCode=255
Jul 28 14:00:29.608167 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.607907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" event={"ID":"70a294fd-3ee6-40f5-b772-5b4d8dca14fd","Type":"ContainerDied","Data":"c7198a87ba641d9ff107b3829f6dfc807845a19c90034afaf3d7b20e32d1b77d"}
Jul 28 14:00:29.608167 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.608047    2579 scope.go:117] "RemoveContainer" containerID="c7198a87ba641d9ff107b3829f6dfc807845a19c90034afaf3d7b20e32d1b77d"
Jul 28 14:00:29.609763 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.609670    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n" event={"ID":"d045014b-e31a-4564-9698-fb0e41bdbecb","Type":"ContainerStarted","Data":"0d01b93ee65a56d303e93219c677fbc74271689719d84fe9ea47676b930bc6a5"}
Jul 28 14:00:29.611413 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.611355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b" event={"ID":"b4915355-1200-44ec-a2ef-2a8ace1c6348","Type":"ContainerStarted","Data":"f8f0998a2ff1049a4964e9f1d1e2a37ba471715af96f1a21bd0490ccaa358cdd"}
Jul 28 14:00:29.613104 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.612900    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw" event={"ID":"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba","Type":"ContainerStarted","Data":"93ff8f34280e8ed99d3506d03efe50256fb100dbc631d883bbe3262fa2e47f9b"}
Jul 28 14:00:29.614577 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.614525    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qqkdd" event={"ID":"943d836b-9abc-4857-9eb8-ea5ae2a76793","Type":"ContainerStarted","Data":"8b4c2f2e0d75165d06ff9b15ee351743c471796f80c213dd154b4dc6bcb3f5c1"}
Jul 28 14:00:29.616257 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.616234    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-crr4w" event={"ID":"c32900e0-0735-40d4-946c-05ec3618add9","Type":"ContainerStarted","Data":"a4c10174585d8c510f195f0b3738acc9cbfa34f00d9b384b654398135a1479b6"}
Jul 28 14:00:29.643227 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.643153    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw" podStartSLOduration=14.4653166 podStartE2EDuration="22.643119146s" totalImagesPullingTime="8.177802546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:29.642268719 +0000 UTC m=+45.769659412" watchObservedRunningTime="2026-07-28 14:00:29.643119146 +0000 UTC m=+45.770509839"
Jul 28 14:00:29.677394 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.676418    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-crr4w" podStartSLOduration=14.515815405 podStartE2EDuration="22.676398197s" totalImagesPullingTime="8.160582792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:29.659452659 +0000 UTC m=+45.786843351" watchObservedRunningTime="2026-07-28 14:00:29.676398197 +0000 UTC m=+45.803788889"
Jul 28 14:00:29.694855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.694798    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qqkdd" podStartSLOduration=2.6081320850000003 podStartE2EDuration="10.694776311s" totalImagesPullingTime="8.086644226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:29.677200171 +0000 UTC m=+45.804590862" watchObservedRunningTime="2026-07-28 14:00:29.694776311 +0000 UTC m=+45.822167004"
Jul 28 14:00:29.694981 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.694909    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b" podStartSLOduration=12.642590535 podStartE2EDuration="20.694901467s" totalImagesPullingTime="8.052310932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:29.694285566 +0000 UTC m=+45.821676268" watchObservedRunningTime="2026-07-28 14:00:29.694901467 +0000 UTC m=+45.822292159"
Jul 28 14:00:29.717520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:29.715701    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqs5n" podStartSLOduration=13.665200088 podStartE2EDuration="21.715678769s" totalImagesPullingTime="8.050478681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:29.714747717 +0000 UTC m=+45.842138411" watchObservedRunningTime="2026-07-28 14:00:29.715678769 +0000 UTC m=+45.843069465"
Jul 28 14:00:30.067469 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.067425    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hwkx9"]
Jul 28 14:00:30.089336 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.089297    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hwkx9"]
Jul 28 14:00:30.089479 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.089433    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.091607 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.091558    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 14:00:30.091755 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.091736    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m7nzk\""
Jul 28 14:00:30.091818 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.091735    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 14:00:30.197641 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.197598    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:30.197815 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.197663    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:30.253116 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.253079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.253265 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.253141    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlc9d\" (UniqueName: \"kubernetes.io/projected/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-api-access-qlc9d\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.253322 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.253307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-data-volume\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.253357 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.253333    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.253392 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.253361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-crio-socket\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.354424 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.354335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-crio-socket\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.354572 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.354432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.354572 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.354474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlc9d\" (UniqueName: \"kubernetes.io/projected/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-api-access-qlc9d\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.354681 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:30.354569    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:00:30.354681 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.354580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-crio-socket\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.354681 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:30.354643    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm podName:7ddcc68d-ba22-42c3-83b8-09e80c6e837d nodeName:}" failed. No retries permitted until 2026-07-28 14:00:30.854622065 +0000 UTC m=+46.982012746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hwkx9" (UID: "7ddcc68d-ba22-42c3-83b8-09e80c6e837d") : configmap "kube-rbac-proxy" not found
Jul 28 14:00:30.354796 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.354755    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-data-volume\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.354796 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.354782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.354922 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:30.354911    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:30.354964 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:30.354956    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls podName:7ddcc68d-ba22-42c3-83b8-09e80c6e837d nodeName:}" failed. No retries permitted until 2026-07-28 14:00:30.854943667 +0000 UTC m=+46.982334340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwkx9" (UID: "7ddcc68d-ba22-42c3-83b8-09e80c6e837d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:30.355063 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.355047    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-data-volume\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.365913 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.365879    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlc9d\" (UniqueName: \"kubernetes.io/projected/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-api-access-qlc9d\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.623605 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.623529    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/1.log"
Jul 28 14:00:30.623998 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.623916    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/0.log"
Jul 28 14:00:30.623998 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.623950    2579 generic.go:356] "Generic (PLEG): container finished" podID="70a294fd-3ee6-40f5-b772-5b4d8dca14fd" containerID="d39b2f58d0d0f4197a712606e4dc87b36a1763c612c67cdc989ae126955daab3" exitCode=255
Jul 28 14:00:30.624142 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.623987    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" event={"ID":"70a294fd-3ee6-40f5-b772-5b4d8dca14fd","Type":"ContainerDied","Data":"d39b2f58d0d0f4197a712606e4dc87b36a1763c612c67cdc989ae126955daab3"}
Jul 28 14:00:30.624142 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.624107    2579 scope.go:117] "RemoveContainer" containerID="c7198a87ba641d9ff107b3829f6dfc807845a19c90034afaf3d7b20e32d1b77d"
Jul 28 14:00:30.624519 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.624493    2579 scope.go:117] "RemoveContainer" containerID="d39b2f58d0d0f4197a712606e4dc87b36a1763c612c67cdc989ae126955daab3"
Jul 28 14:00:30.624697 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:30.624674    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-nhmps_openshift-console-operator(70a294fd-3ee6-40f5-b772-5b4d8dca14fd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" podUID="70a294fd-3ee6-40f5-b772-5b4d8dca14fd"
Jul 28 14:00:30.858973 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.858944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.859164 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:30.859009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:30.859164 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:30.859109    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:00:30.859164 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:30.859110    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:30.859267 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:30.859169    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm podName:7ddcc68d-ba22-42c3-83b8-09e80c6e837d nodeName:}" failed. No retries permitted until 2026-07-28 14:00:31.859152631 +0000 UTC m=+47.986543323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hwkx9" (UID: "7ddcc68d-ba22-42c3-83b8-09e80c6e837d") : configmap "kube-rbac-proxy" not found
Jul 28 14:00:30.859267 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:30.859183    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls podName:7ddcc68d-ba22-42c3-83b8-09e80c6e837d nodeName:}" failed. No retries permitted until 2026-07-28 14:00:31.859176926 +0000 UTC m=+47.986567595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwkx9" (UID: "7ddcc68d-ba22-42c3-83b8-09e80c6e837d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:31.628435 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:31.628413    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/1.log"
Jul 28 14:00:31.628803 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:31.628731    2579 scope.go:117] "RemoveContainer" containerID="d39b2f58d0d0f4197a712606e4dc87b36a1763c612c67cdc989ae126955daab3"
Jul 28 14:00:31.628889 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:31.628871    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-nhmps_openshift-console-operator(70a294fd-3ee6-40f5-b772-5b4d8dca14fd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" podUID="70a294fd-3ee6-40f5-b772-5b4d8dca14fd"
Jul 28 14:00:31.865829 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:31.865799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:31.865968 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:31.865865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:31.865968 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:31.865953    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:31.866067 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:31.865979    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:00:31.866067 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:31.866046    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls podName:7ddcc68d-ba22-42c3-83b8-09e80c6e837d nodeName:}" failed. No retries permitted until 2026-07-28 14:00:33.866007738 +0000 UTC m=+49.993398410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwkx9" (UID: "7ddcc68d-ba22-42c3-83b8-09e80c6e837d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:31.866067 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:31.866067    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm podName:7ddcc68d-ba22-42c3-83b8-09e80c6e837d nodeName:}" failed. No retries permitted until 2026-07-28 14:00:33.866056268 +0000 UTC m=+49.993446940 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hwkx9" (UID: "7ddcc68d-ba22-42c3-83b8-09e80c6e837d") : configmap "kube-rbac-proxy" not found
Jul 28 14:00:33.050240 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:33.050204    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zfd6l_68fa69ca-ef93-41a4-9b62-14e2804b3e26/dns-node-resolver/0.log"
Jul 28 14:00:33.646889 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:33.646859    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9df2t_d510aeee-4e32-4319-a8c6-a47ba4ae9205/node-ca/0.log"
Jul 28 14:00:33.883894 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:33.883857    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:33.884230 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:33.883977    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:00:33.884230 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:33.884012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:33.884230 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:33.884052    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm podName:7ddcc68d-ba22-42c3-83b8-09e80c6e837d nodeName:}" failed. No retries permitted until 2026-07-28 14:00:37.884036213 +0000 UTC m=+54.011426885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hwkx9" (UID: "7ddcc68d-ba22-42c3-83b8-09e80c6e837d") : configmap "kube-rbac-proxy" not found
Jul 28 14:00:33.884230 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:33.884103    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:33.884230 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:33.884176    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls podName:7ddcc68d-ba22-42c3-83b8-09e80c6e837d nodeName:}" failed. No retries permitted until 2026-07-28 14:00:37.884160818 +0000 UTC m=+54.011551489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwkx9" (UID: "7ddcc68d-ba22-42c3-83b8-09e80c6e837d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:34.647198 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:34.647172    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qqkdd_943d836b-9abc-4857-9eb8-ea5ae2a76793/serve-healthcheck-canary/0.log"
Jul 28 14:00:35.448699 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:35.448669    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw_85a8fcc2-28d8-4534-908a-c0ea8fdf51ba/kube-storage-version-migrator-operator/0.log"
Jul 28 14:00:35.598037 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:35.597987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:35.598221 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.598149    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 14:00:35.598221 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.598168    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7657b854fb-p5zb5: secret "image-registry-tls" not found
Jul 28 14:00:35.598326 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.598234    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls podName:7291bf28-e6bd-4c59-983e-78844bef303e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:51.598213508 +0000 UTC m=+67.725604194 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls") pod "image-registry-7657b854fb-p5zb5" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e") : secret "image-registry-tls" not found
Jul 28 14:00:35.699224 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:35.699152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:35.699224 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:35.699201    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:35.699574 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:35.699245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:35.699574 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.699307    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:51.699290799 +0000 UTC m=+67.826681468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : configmap references non-existent config key: service-ca.crt
Jul 28 14:00:35.699574 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.699372    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:35.699574 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.699374    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 14:00:35.699574 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.699428    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls podName:97a45da1-bd05-4efe-b270-1f7a8bff838e nodeName:}" failed. No retries permitted until 2026-07-28 14:00:51.699411785 +0000 UTC m=+67.826802469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-2rhml" (UID: "97a45da1-bd05-4efe-b270-1f7a8bff838e") : secret "cluster-monitoring-operator-tls" not found
Jul 28 14:00:35.699574 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.699443    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs podName:0a4f01d9-d677-4b3c-8d48-7076c851f2fd nodeName:}" failed. No retries permitted until 2026-07-28 14:00:51.699436595 +0000 UTC m=+67.826827264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs") pod "router-default-7c97498d97-j9zq7" (UID: "0a4f01d9-d677-4b3c-8d48-7076c851f2fd") : secret "router-metrics-certs-default" not found
Jul 28 14:00:35.800029 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:35.799988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:35.800150 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:35.800036    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:35.800150 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.800122    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:00:35.800219 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.800152    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:00:35.800219 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.800175    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls podName:1e4ec977-d8d3-4bd6-9bb0-eb5df490604c nodeName:}" failed. No retries permitted until 2026-07-28 14:00:51.800159607 +0000 UTC m=+67.927550292 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls") pod "dns-default-khjf6" (UID: "1e4ec977-d8d3-4bd6-9bb0-eb5df490604c") : secret "dns-default-metrics-tls" not found
Jul 28 14:00:35.800219 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:35.800206    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls podName:5833ae33-c691-4b49-a25b-d221280278d8 nodeName:}" failed. No retries permitted until 2026-07-28 14:00:51.800195894 +0000 UTC m=+67.927586563 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls") pod "cluster-samples-operator-6748489656-fkksj" (UID: "5833ae33-c691-4b49-a25b-d221280278d8") : secret "samples-operator-tls" not found
Jul 28 14:00:37.917586 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:37.917554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:37.918162 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:37.917616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:37.918162 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:37.917732    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:37.918162 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:37.917815    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls podName:7ddcc68d-ba22-42c3-83b8-09e80c6e837d nodeName:}" failed. No retries permitted until 2026-07-28 14:00:45.917793976 +0000 UTC m=+62.045184662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hwkx9" (UID: "7ddcc68d-ba22-42c3-83b8-09e80c6e837d") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:00:37.918162 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:37.918115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:40.197027 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:40.196990    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:40.197511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:40.197062    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:40.197581 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:40.197522    2579 scope.go:117] "RemoveContainer" containerID="d39b2f58d0d0f4197a712606e4dc87b36a1763c612c67cdc989ae126955daab3"
Jul 28 14:00:40.657098 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:40.657073    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:00:40.657520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:40.657501    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/1.log"
Jul 28 14:00:40.657645 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:40.657539    2579 generic.go:356] "Generic (PLEG): container finished" podID="70a294fd-3ee6-40f5-b772-5b4d8dca14fd" containerID="8969646fd056c4c303d98523c8e40e39d1d8e5631c28b1910fbcb612b8d47afe" exitCode=255
Jul 28 14:00:40.657645 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:40.657615    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" event={"ID":"70a294fd-3ee6-40f5-b772-5b4d8dca14fd","Type":"ContainerDied","Data":"8969646fd056c4c303d98523c8e40e39d1d8e5631c28b1910fbcb612b8d47afe"}
Jul 28 14:00:40.657719 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:40.657670    2579 scope.go:117] "RemoveContainer" containerID="d39b2f58d0d0f4197a712606e4dc87b36a1763c612c67cdc989ae126955daab3"
Jul 28 14:00:40.658007 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:40.657983    2579 scope.go:117] "RemoveContainer" containerID="8969646fd056c4c303d98523c8e40e39d1d8e5631c28b1910fbcb612b8d47afe"
Jul 28 14:00:40.658186 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:40.658168    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-nhmps_openshift-console-operator(70a294fd-3ee6-40f5-b772-5b4d8dca14fd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" podUID="70a294fd-3ee6-40f5-b772-5b4d8dca14fd"
Jul 28 14:00:41.662476 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:41.662449    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:00:44.531147 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:44.531115    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m2b9r"
Jul 28 14:00:45.985352 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:45.985316    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:45.987686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:45.987663    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7ddcc68d-ba22-42c3-83b8-09e80c6e837d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hwkx9\" (UID: \"7ddcc68d-ba22-42c3-83b8-09e80c6e837d\") " pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:46.002244 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:46.002220    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m7nzk\""
Jul 28 14:00:46.009954 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:46.009941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hwkx9"
Jul 28 14:00:46.142159 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:46.142131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hwkx9"]
Jul 28 14:00:46.145391 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:46.145362    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ddcc68d_ba22_42c3_83b8_09e80c6e837d.slice/crio-f92c37742117cb1d03ea190ca6faf85b64d02c870a748f8f0bf5911c38c2a398 WatchSource:0}: Error finding container f92c37742117cb1d03ea190ca6faf85b64d02c870a748f8f0bf5911c38c2a398: Status 404 returned error can't find the container with id f92c37742117cb1d03ea190ca6faf85b64d02c870a748f8f0bf5911c38c2a398
Jul 28 14:00:46.678920 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:46.678884    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hwkx9" event={"ID":"7ddcc68d-ba22-42c3-83b8-09e80c6e837d","Type":"ContainerStarted","Data":"f92c37742117cb1d03ea190ca6faf85b64d02c870a748f8f0bf5911c38c2a398"}
Jul 28 14:00:50.125532 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.125493    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:50.125990 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.125548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:50.128370 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.128349    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 14:00:50.128978 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.128962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 14:00:50.138302 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.138280    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e2f0a8fb-0a74-4e58-8195-47b0ac92551b-original-pull-secret\") pod \"global-pull-secret-syncer-c75dk\" (UID: \"e2f0a8fb-0a74-4e58-8195-47b0ac92551b\") " pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:50.138422 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.138300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66bf6b5d-9493-4aaf-aa4f-2f3788bf51be-metrics-certs\") pod \"network-metrics-daemon-8ltnr\" (UID: \"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be\") " pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:50.151353 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.151324    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75dk"
Jul 28 14:00:50.164400 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.164378    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-96fnv\""
Jul 28 14:00:50.172594 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.172556    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltnr"
Jul 28 14:00:50.196946 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.196919    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:50.197084 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.196986    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:00:50.197350 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.197333    2579 scope.go:117] "RemoveContainer" containerID="8969646fd056c4c303d98523c8e40e39d1d8e5631c28b1910fbcb612b8d47afe"
Jul 28 14:00:50.197561 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:00:50.197530    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-nhmps_openshift-console-operator(70a294fd-3ee6-40f5-b772-5b4d8dca14fd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" podUID="70a294fd-3ee6-40f5-b772-5b4d8dca14fd"
Jul 28 14:00:50.226604 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.226570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:50.229508 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.229480    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4knh\" (UniqueName: \"kubernetes.io/projected/7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc-kube-api-access-n4knh\") pod \"network-check-target-shngl\" (UID: \"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc\") " pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:50.472223 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.472147    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vgxlf\""
Jul 28 14:00:50.479619 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:50.479597    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:51.638417 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.638389    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:51.641194 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.641172    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"image-registry-7657b854fb-p5zb5\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") " pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:51.739922 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.739469    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:51.739922 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.739529    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:51.739922 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.739569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:51.740224 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.740203    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-service-ca-bundle\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:51.743070 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.742989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97a45da1-bd05-4efe-b270-1f7a8bff838e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-2rhml\" (UID: \"97a45da1-bd05-4efe-b270-1f7a8bff838e\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:51.743921 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.743884    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a4f01d9-d677-4b3c-8d48-7076c851f2fd-metrics-certs\") pod \"router-default-7c97498d97-j9zq7\" (UID: \"0a4f01d9-d677-4b3c-8d48-7076c851f2fd\") " pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:51.775960 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.775881    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8ltnr"]
Jul 28 14:00:51.784608 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:51.784581    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66bf6b5d_9493_4aaf_aa4f_2f3788bf51be.slice/crio-c6782f05806497f2343f2d7fb74c1b21678bced52e7312622edfcb62d0a6afa7 WatchSource:0}: Error finding container c6782f05806497f2343f2d7fb74c1b21678bced52e7312622edfcb62d0a6afa7: Status 404 returned error can't find the container with id c6782f05806497f2343f2d7fb74c1b21678bced52e7312622edfcb62d0a6afa7
Jul 28 14:00:51.839955 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.839933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:51.840049 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.839982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:51.842341 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.842320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e4ec977-d8d3-4bd6-9bb0-eb5df490604c-metrics-tls\") pod \"dns-default-khjf6\" (UID: \"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c\") " pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:51.842490 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.842466    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5833ae33-c691-4b49-a25b-d221280278d8-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-fkksj\" (UID: \"5833ae33-c691-4b49-a25b-d221280278d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:51.895252 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.895191    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mljsl\""
Jul 28 14:00:51.903390 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.903374    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:51.937941 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.937917    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zzwrj\""
Jul 28 14:00:51.946357 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.946334    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"
Jul 28 14:00:51.960711 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.960693    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j8n4c\""
Jul 28 14:00:51.968660 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:51.968630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:52.004331 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.004283    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c75dk"]
Jul 28 14:00:52.007469 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.007196    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-shngl"]
Jul 28 14:00:52.019454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.019405    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f74xg\""
Jul 28 14:00:52.027205 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.027180    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"
Jul 28 14:00:52.033431 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.033412    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4vfd5\""
Jul 28 14:00:52.040954 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.040929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-khjf6"
Jul 28 14:00:52.072651 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.072614    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7657b854fb-p5zb5"]
Jul 28 14:00:52.122771 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.122664    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml"]
Jul 28 14:00:52.133519 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:52.133490    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97a45da1_bd05_4efe_b270_1f7a8bff838e.slice/crio-9eff4a917a4ce8eb6805486d641c60eeb70e8dc9cc7a3065def4c9efabf19478 WatchSource:0}: Error finding container 9eff4a917a4ce8eb6805486d641c60eeb70e8dc9cc7a3065def4c9efabf19478: Status 404 returned error can't find the container with id 9eff4a917a4ce8eb6805486d641c60eeb70e8dc9cc7a3065def4c9efabf19478
Jul 28 14:00:52.155993 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.155968    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c97498d97-j9zq7"]
Jul 28 14:00:52.160316 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:52.160278    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4f01d9_d677_4b3c_8d48_7076c851f2fd.slice/crio-322ddb29a86549bb05ad316a0e1bd06225ec68586d16b3c1b6c3048bde20b2f4 WatchSource:0}: Error finding container 322ddb29a86549bb05ad316a0e1bd06225ec68586d16b3c1b6c3048bde20b2f4: Status 404 returned error can't find the container with id 322ddb29a86549bb05ad316a0e1bd06225ec68586d16b3c1b6c3048bde20b2f4
Jul 28 14:00:52.202255 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.202229    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj"]
Jul 28 14:00:52.224962 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.224932    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-khjf6"]
Jul 28 14:00:52.228271 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:00:52.228244    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e4ec977_d8d3_4bd6_9bb0_eb5df490604c.slice/crio-31b719237e36ba1653fc980d4b2ab177ff436fc09a15c21d7487086c8155f068 WatchSource:0}: Error finding container 31b719237e36ba1653fc980d4b2ab177ff436fc09a15c21d7487086c8155f068: Status 404 returned error can't find the container with id 31b719237e36ba1653fc980d4b2ab177ff436fc09a15c21d7487086c8155f068
Jul 28 14:00:52.709294 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.709254    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hwkx9" event={"ID":"7ddcc68d-ba22-42c3-83b8-09e80c6e837d","Type":"ContainerStarted","Data":"7d516b8db8da34cd207b0b74de817b8aa10efad17f7dc660e417db62580a40fc"}
Jul 28 14:00:52.711170 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.711079    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-khjf6" event={"ID":"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c","Type":"ContainerStarted","Data":"31b719237e36ba1653fc980d4b2ab177ff436fc09a15c21d7487086c8155f068"}
Jul 28 14:00:52.713882 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.713854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj" event={"ID":"5833ae33-c691-4b49-a25b-d221280278d8","Type":"ContainerStarted","Data":"51e755f5a1c01deba26a46c9df54b148f63be354f7d3226c9ebd9fc91f9cb702"}
Jul 28 14:00:52.717050 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.716997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5" event={"ID":"7291bf28-e6bd-4c59-983e-78844bef303e","Type":"ContainerStarted","Data":"0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c"}
Jul 28 14:00:52.717168 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.717074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5" event={"ID":"7291bf28-e6bd-4c59-983e-78844bef303e","Type":"ContainerStarted","Data":"78583ff66cda9ebc6bb3043afe982daa11188f48ea874e7db717e67916b173ee"}
Jul 28 14:00:52.717393 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.717374    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:00:52.718675 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.718620    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8ltnr" event={"ID":"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be","Type":"ContainerStarted","Data":"c6782f05806497f2343f2d7fb74c1b21678bced52e7312622edfcb62d0a6afa7"}
Jul 28 14:00:52.720541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.720272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c75dk" event={"ID":"e2f0a8fb-0a74-4e58-8195-47b0ac92551b","Type":"ContainerStarted","Data":"20f94081614a7eb3234f48a00a9369f752bb39cf4bc691ece850cc59adb39fe8"}
Jul 28 14:00:52.724875 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.724306    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c97498d97-j9zq7" event={"ID":"0a4f01d9-d677-4b3c-8d48-7076c851f2fd","Type":"ContainerStarted","Data":"1a44586e583ee777d637ede1319de4e0ecd833e39ad44c98a5226783f36f0917"}
Jul 28 14:00:52.724875 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.724369    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c97498d97-j9zq7" event={"ID":"0a4f01d9-d677-4b3c-8d48-7076c851f2fd","Type":"ContainerStarted","Data":"322ddb29a86549bb05ad316a0e1bd06225ec68586d16b3c1b6c3048bde20b2f4"}
Jul 28 14:00:52.728096 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.728033    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml" event={"ID":"97a45da1-bd05-4efe-b270-1f7a8bff838e","Type":"ContainerStarted","Data":"9eff4a917a4ce8eb6805486d641c60eeb70e8dc9cc7a3065def4c9efabf19478"}
Jul 28 14:00:52.729903 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.729880    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-shngl" event={"ID":"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc","Type":"ContainerStarted","Data":"da80858b500fd8bdf714061aa53601d023a530e83835e4617dc792ffbc4034a2"}
Jul 28 14:00:52.729997 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.729923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-shngl" event={"ID":"7ceeeedc-b2e7-47c4-a7f7-697bcb1babcc","Type":"ContainerStarted","Data":"d23700359b2406534c993d8709122e03c8a0ee0e696573f6767757678c160f02"}
Jul 28 14:00:52.730438 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.730416    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:00:52.765262 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.764216    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5" podStartSLOduration=67.764193887 podStartE2EDuration="1m7.764193887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:52.739809649 +0000 UTC m=+68.867200349" watchObservedRunningTime="2026-07-28 14:00:52.764193887 +0000 UTC m=+68.891584579"
Jul 28 14:00:52.794377 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.793952    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-shngl" podStartSLOduration=67.793929456 podStartE2EDuration="1m7.793929456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:52.79280495 +0000 UTC m=+68.920195644" watchObservedRunningTime="2026-07-28 14:00:52.793929456 +0000 UTC m=+68.921320127"
Jul 28 14:00:52.794377 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.794240    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7c97498d97-j9zq7" podStartSLOduration=45.794227941 podStartE2EDuration="45.794227941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:52.765117134 +0000 UTC m=+68.892507826" watchObservedRunningTime="2026-07-28 14:00:52.794227941 +0000 UTC m=+68.921618630"
Jul 28 14:00:52.969110 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.969003    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:52.972200 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:52.972173    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:53.740042 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:53.739344    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8ltnr" event={"ID":"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be","Type":"ContainerStarted","Data":"32cc69e124eb33f73572cc96be3a127ba9cfa75f09536bba4476e8169c81391b"}
Jul 28 14:00:53.740042 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:53.739412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8ltnr" event={"ID":"66bf6b5d-9493-4aaf-aa4f-2f3788bf51be","Type":"ContainerStarted","Data":"45730e410dfa63d4b04ac0871f6cc2168cbe9af765afa1e974800188a19d972f"}
Jul 28 14:00:53.740830 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:53.740779    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:53.742305 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:53.742281    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7c97498d97-j9zq7"
Jul 28 14:00:53.787660 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:00:53.787602    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8ltnr" podStartSLOduration=67.352778412 podStartE2EDuration="1m8.787578085s" totalImagesPullingTime="1.434799673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:00:53.762816634 +0000 UTC m=+69.890207328" watchObservedRunningTime="2026-07-28 14:00:53.787578085 +0000 UTC m=+69.914968777"
Jul 28 14:01:00.766282 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.766242    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hwkx9" event={"ID":"7ddcc68d-ba22-42c3-83b8-09e80c6e837d","Type":"ContainerStarted","Data":"9a3875ff426eb66dd5d2ea64927d0b288eccee1bd34b9a122a38a3b9ca33a820"}
Jul 28 14:01:00.767811 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.767785    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-khjf6" event={"ID":"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c","Type":"ContainerStarted","Data":"8451fcbcac58379bac7851a9675ab0d9f4ff4639a002666e3c58ac6ff3458114"}
Jul 28 14:01:00.767917 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.767837    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-khjf6" event={"ID":"1e4ec977-d8d3-4bd6-9bb0-eb5df490604c","Type":"ContainerStarted","Data":"7b20a872b9e5c47ee1fae8a8ea9669b06f87275f7a38198ec278f42eefe6cef6"}
Jul 28 14:01:00.767917 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.767903    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-khjf6"
Jul 28 14:01:00.769531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.769503    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj" event={"ID":"5833ae33-c691-4b49-a25b-d221280278d8","Type":"ContainerStarted","Data":"e26adab7f1d28e277fb235cd9082a1f870ff1945584e7127660f0bbe54a0fbfd"}
Jul 28 14:01:00.769669 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.769552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj" event={"ID":"5833ae33-c691-4b49-a25b-d221280278d8","Type":"ContainerStarted","Data":"4b84f90d0d633e650df5af27a6b6c98fad9ca68f034874896deedbabbb6d323a"}
Jul 28 14:01:00.770927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.770906    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c75dk" event={"ID":"e2f0a8fb-0a74-4e58-8195-47b0ac92551b","Type":"ContainerStarted","Data":"138773f9271b97418f43a6d93988dd3732774dc6fa5d1e6c06e5be46249e3ca5"}
Jul 28 14:01:00.772286 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.772267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml" event={"ID":"97a45da1-bd05-4efe-b270-1f7a8bff838e","Type":"ContainerStarted","Data":"2542ebbd3ee28d7c6c6c40d854ab8360c8bff9767446e676cde79b77fb3e6564"}
Jul 28 14:01:00.787390 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.787354    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-khjf6" podStartSLOduration=34.23982233 podStartE2EDuration="41.787341861s" totalImagesPullingTime="7.547519531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:00.785792165 +0000 UTC m=+76.913182856" watchObservedRunningTime="2026-07-28 14:01:00.787341861 +0000 UTC m=+76.914732551"
Jul 28 14:01:00.801268 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.801224    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c75dk" podStartSLOduration=68.044934805 podStartE2EDuration="1m15.801211838s" totalImagesPullingTime="7.756277033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:00.800314216 +0000 UTC m=+76.927704907" watchObservedRunningTime="2026-07-28 14:01:00.801211838 +0000 UTC m=+76.928602580"
Jul 28 14:01:00.818783 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.818745    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-2rhml" podStartSLOduration=46.182187627 podStartE2EDuration="53.818728172s" totalImagesPullingTime="7.636540545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:00.816925515 +0000 UTC m=+76.944316218" watchObservedRunningTime="2026-07-28 14:01:00.818728172 +0000 UTC m=+76.946118866"
Jul 28 14:01:00.832425 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:00.832376    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-fkksj" podStartSLOduration=46.301828335 podStartE2EDuration="53.832364599s" totalImagesPullingTime="7.530536264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:00.831961848 +0000 UTC m=+76.959352542" watchObservedRunningTime="2026-07-28 14:01:00.832364599 +0000 UTC m=+76.959755290"
Jul 28 14:01:01.336641 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:01.336608    2579 scope.go:117] "RemoveContainer" containerID="8969646fd056c4c303d98523c8e40e39d1d8e5631c28b1910fbcb612b8d47afe"
Jul 28 14:01:01.779304 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:01.779277    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:01:01.779761 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:01.779485    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" event={"ID":"70a294fd-3ee6-40f5-b772-5b4d8dca14fd","Type":"ContainerStarted","Data":"32dfcd7e7efc79bcde4d98c57a89e99a1f3f4e36a5711f684c3fc134e4169f28"}
Jul 28 14:01:01.799717 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:01.799658    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps" podStartSLOduration=46.67795517 podStartE2EDuration="54.799634462s" totalImagesPullingTime="8.121679292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:01.798588462 +0000 UTC m=+77.925979173" watchObservedRunningTime="2026-07-28 14:01:01.799634462 +0000 UTC m=+77.927025155"
Jul 28 14:01:04.792710 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:04.792605    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hwkx9" event={"ID":"7ddcc68d-ba22-42c3-83b8-09e80c6e837d","Type":"ContainerStarted","Data":"c5ae8b581f0ed0a32df92e379fb4c5d86eaeea14a26ead7dab813a9fdee9efc0"}
Jul 28 14:01:04.812855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:04.812806    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hwkx9" podStartSLOduration=16.392108766 podStartE2EDuration="34.812785242s" totalImagesPullingTime="18.420676476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:00:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:04.812696708 +0000 UTC m=+80.940087411" watchObservedRunningTime="2026-07-28 14:01:04.812785242 +0000 UTC m=+80.940175937"
Jul 28 14:01:10.781563 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:10.781531    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-khjf6"
Jul 28 14:01:11.780219 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:11.780187    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:01:11.783839 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:11.783822    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-nhmps"
Jul 28 14:01:13.745201 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.745173    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:01:13.913123 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.913098    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"]
Jul 28 14:01:13.917597 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.917572    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:13.920047 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.920011    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 14:01:13.920047 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.920040    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 14:01:13.920244 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.920086    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ktw9p\""
Jul 28 14:01:13.920294 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.920250    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 14:01:13.922188 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.922170    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 14:01:13.933498 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.933434    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"]
Jul 28 14:01:13.948037 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.945081    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-52cwh"]
Jul 28 14:01:13.957937 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.948894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:13.958288 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.958269    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 14:01:13.958390 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.958296    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nh4t2\""
Jul 28 14:01:13.958463 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.958421    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 14:01:13.958595 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:13.958565    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 14:01:14.022298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.022268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/723bc418-d780-4e89-b5ad-89dbe94ffda5-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.022447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.022308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.022447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.022332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.022447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.022400    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djm62\" (UniqueName: \"kubernetes.io/projected/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-api-access-djm62\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.022552 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.022460    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/723bc418-d780-4e89-b5ad-89dbe94ffda5-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.022552 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.022480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.123589 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.123747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/723bc418-d780-4e89-b5ad-89dbe94ffda5-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.123747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63d71a17-a444-4fba-af29-fe4217c8c252-metrics-client-ca\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.123747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.123747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdmm6\" (UniqueName: \"kubernetes.io/projected/63d71a17-a444-4fba-af29-fe4217c8c252-kube-api-access-xdmm6\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.123747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-textfile\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.123747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-wtmp\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.124381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.124381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djm62\" (UniqueName: \"kubernetes.io/projected/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-api-access-djm62\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.124381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123874    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/723bc418-d780-4e89-b5ad-89dbe94ffda5-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.124381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.124381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-tls\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.124381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/723bc418-d780-4e89-b5ad-89dbe94ffda5-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.124381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.123995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-accelerators-collector-config\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.124381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.124098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63d71a17-a444-4fba-af29-fe4217c8c252-sys\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.124381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.124149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/63d71a17-a444-4fba-af29-fe4217c8c252-root\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.124843 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.124586    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/723bc418-d780-4e89-b5ad-89dbe94ffda5-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.125080 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.125060    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.126310 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.126293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.126465 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.126448    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.133735 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.133716    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djm62\" (UniqueName: \"kubernetes.io/projected/723bc418-d780-4e89-b5ad-89dbe94ffda5-kube-api-access-djm62\") pod \"kube-state-metrics-56d9f944b4-7w4jd\" (UID: \"723bc418-d780-4e89-b5ad-89dbe94ffda5\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.225357 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225325    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-tls\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225497 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-accelerators-collector-config\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225497 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63d71a17-a444-4fba-af29-fe4217c8c252-sys\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225497 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225417    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/63d71a17-a444-4fba-af29-fe4217c8c252-root\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225497 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225477    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/63d71a17-a444-4fba-af29-fe4217c8c252-root\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225673 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63d71a17-a444-4fba-af29-fe4217c8c252-sys\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225673 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225673 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63d71a17-a444-4fba-af29-fe4217c8c252-metrics-client-ca\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225673 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225565    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdmm6\" (UniqueName: \"kubernetes.io/projected/63d71a17-a444-4fba-af29-fe4217c8c252-kube-api-access-xdmm6\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225673 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-textfile\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225883 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-wtmp\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225937 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225880    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-wtmp\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.225937 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.225907    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-textfile\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.226085 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.226062    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-accelerators-collector-config\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.226138 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.226092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63d71a17-a444-4fba-af29-fe4217c8c252-metrics-client-ca\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.226180 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.226168    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"
Jul 28 14:01:14.227837 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.227816    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.227938 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.227921    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63d71a17-a444-4fba-af29-fe4217c8c252-node-exporter-tls\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.236233 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.236212    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdmm6\" (UniqueName: \"kubernetes.io/projected/63d71a17-a444-4fba-af29-fe4217c8c252-kube-api-access-xdmm6\") pod \"node-exporter-52cwh\" (UID: \"63d71a17-a444-4fba-af29-fe4217c8c252\") " pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.265752 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.265736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-52cwh"
Jul 28 14:01:14.278205 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:01:14.278178    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d71a17_a444_4fba_af29_fe4217c8c252.slice/crio-ddd6bf060d8b58e6365082535139b3fa5b752b98d41c23979ba05232cb5e7425 WatchSource:0}: Error finding container ddd6bf060d8b58e6365082535139b3fa5b752b98d41c23979ba05232cb5e7425: Status 404 returned error can't find the container with id ddd6bf060d8b58e6365082535139b3fa5b752b98d41c23979ba05232cb5e7425
Jul 28 14:01:14.375830 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.375810    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd"]
Jul 28 14:01:14.377638 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:01:14.377611    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723bc418_d780_4e89_b5ad_89dbe94ffda5.slice/crio-7a2496c2e377c99d3c04700118c07f3909327582f0273a32a23d822fed3260f0 WatchSource:0}: Error finding container 7a2496c2e377c99d3c04700118c07f3909327582f0273a32a23d822fed3260f0: Status 404 returned error can't find the container with id 7a2496c2e377c99d3c04700118c07f3909327582f0273a32a23d822fed3260f0
Jul 28 14:01:14.829898 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.829863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-52cwh" event={"ID":"63d71a17-a444-4fba-af29-fe4217c8c252","Type":"ContainerStarted","Data":"ddd6bf060d8b58e6365082535139b3fa5b752b98d41c23979ba05232cb5e7425"}
Jul 28 14:01:14.830992 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:14.830965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd" event={"ID":"723bc418-d780-4e89-b5ad-89dbe94ffda5","Type":"ContainerStarted","Data":"7a2496c2e377c99d3c04700118c07f3909327582f0273a32a23d822fed3260f0"}
Jul 28 14:01:15.005887 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.005859    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:01:15.010549 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.010532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.018098 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018074    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 14:01:15.018098 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018089    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 14:01:15.018354 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018340    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 14:01:15.018455 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018418    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 14:01:15.018543 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018525    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dl28v\""
Jul 28 14:01:15.018595 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018584    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 14:01:15.018689 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018676    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 14:01:15.018738 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018701    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 14:01:15.018738 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018722    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 14:01:15.018828 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.018743    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 14:01:15.029682 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.029655    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:01:15.131906 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.131815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.131906 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.131868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132106 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.131942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132106 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.131974    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132106 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.131996    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132106 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.132012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52jzv\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-kube-api-access-52jzv\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.132103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.132143    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.132170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-web-config\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.132205    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132383 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.132246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132383 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.132291    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.132383 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.132320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-config-out\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.232758 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.232721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.232927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.232770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-config-out\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.232927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.232808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.232927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.232834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.232927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.232867    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.232927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.232896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.232927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.232921    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.233256 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.232944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52jzv\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-kube-api-access-52jzv\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.233256 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.232981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.233256 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.233011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.233256 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.233065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-web-config\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.233256 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.233095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.233256 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.233132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.233951 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.233925    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.234546 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.234238    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.235767 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.235739    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.236189 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.236164    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.236280 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.236259    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.236780 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.236755    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.236885 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.236793    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-web-config\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.236885 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.236861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-config-out\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.237082 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.237004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.237813 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.237792    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.238732 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.238705    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.239106 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.239087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.244592 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.244571    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52jzv\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-kube-api-access-52jzv\") pod \"alertmanager-main-0\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.319863 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.319836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:01:15.458372 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.458333    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:01:15.462142 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:01:15.462117    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c084083_aade_4fb9_854c_99fc8700fed6.slice/crio-23bb16ba44825dc76318398ba7fbf909c3c24623a26a6c07348c5ae18864a7e3 WatchSource:0}: Error finding container 23bb16ba44825dc76318398ba7fbf909c3c24623a26a6c07348c5ae18864a7e3: Status 404 returned error can't find the container with id 23bb16ba44825dc76318398ba7fbf909c3c24623a26a6c07348c5ae18864a7e3
Jul 28 14:01:15.837665 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:15.837622    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerStarted","Data":"23bb16ba44825dc76318398ba7fbf909c3c24623a26a6c07348c5ae18864a7e3"}
Jul 28 14:01:16.843497 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.843464    2579 generic.go:356] "Generic (PLEG): container finished" podID="63d71a17-a444-4fba-af29-fe4217c8c252" containerID="c5fa185cc5164963b5d7f7cd2dd7c8f3f680b01152858ceb628cc620738a75db" exitCode=0
Jul 28 14:01:16.843900 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.843529    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-52cwh" event={"ID":"63d71a17-a444-4fba-af29-fe4217c8c252","Type":"ContainerDied","Data":"c5fa185cc5164963b5d7f7cd2dd7c8f3f680b01152858ceb628cc620738a75db"}
Jul 28 14:01:16.857828 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.857804    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"]
Jul 28 14:01:16.861990 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.861972    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:16.864536 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.864513    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 14:01:16.864653 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.864513    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 14:01:16.864909 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.864808    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-g9kct\""
Jul 28 14:01:16.864909 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.864849    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 14:01:16.864909 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.864893    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 14:01:16.865286 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.865266    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 14:01:16.865371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.865310    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8lpe4a3fhq448\""
Jul 28 14:01:16.880391 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.880364    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"]
Jul 28 14:01:16.950585 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.950552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-metrics-client-ca\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:16.950749 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.950607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:16.950749 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.950638    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-grpc-tls\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:16.950749 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.950667    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-tls\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:16.950749 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.950694    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:16.950979 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.950794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:16.950979 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.950861    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:16.950979 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:16.950892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkfmr\" (UniqueName: \"kubernetes.io/projected/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-kube-api-access-nkfmr\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.051945 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.051920    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-metrics-client-ca\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.052066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.051956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.052066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.051973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-grpc-tls\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.052066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.051990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-tls\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.052066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.052007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.052066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.052053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.052336 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.052085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.052336 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.052113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkfmr\" (UniqueName: \"kubernetes.io/projected/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-kube-api-access-nkfmr\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.053079 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.053051    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-metrics-client-ca\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.055342 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.055260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.055342 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.055295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.055496 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.055343    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.055496 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.055393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.055496 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.055477    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-thanos-querier-tls\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.055848 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.055828    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-secret-grpc-tls\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.061965 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.061850    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkfmr\" (UniqueName: \"kubernetes.io/projected/91b2b8f5-583d-4396-bbdb-0c96b3150ea3-kube-api-access-nkfmr\") pod \"thanos-querier-6f984bcdc7-qjj8h\" (UID: \"91b2b8f5-583d-4396-bbdb-0c96b3150ea3\") " pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.175227 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.173733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:17.370698 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.370606    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"]
Jul 28 14:01:17.374367 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:01:17.374332    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b2b8f5_583d_4396_bbdb_0c96b3150ea3.slice/crio-a78af4e83be008258d7732390dcc34bff672d927f7b078ed59fa5b9161e3052e WatchSource:0}: Error finding container a78af4e83be008258d7732390dcc34bff672d927f7b078ed59fa5b9161e3052e: Status 404 returned error can't find the container with id a78af4e83be008258d7732390dcc34bff672d927f7b078ed59fa5b9161e3052e
Jul 28 14:01:17.848699 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.848674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h" event={"ID":"91b2b8f5-583d-4396-bbdb-0c96b3150ea3","Type":"ContainerStarted","Data":"a78af4e83be008258d7732390dcc34bff672d927f7b078ed59fa5b9161e3052e"}
Jul 28 14:01:17.850937 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.850911    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-52cwh" event={"ID":"63d71a17-a444-4fba-af29-fe4217c8c252","Type":"ContainerStarted","Data":"b645df2751636e87d44d2f0ea83eddcfec06a4717032e59eaf2bfeccf25799f0"}
Jul 28 14:01:17.851050 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.850965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-52cwh" event={"ID":"63d71a17-a444-4fba-af29-fe4217c8c252","Type":"ContainerStarted","Data":"39466ed9adbf1f92b630bddc9c735295fd148a957f4968b18c87089b3413b059"}
Jul 28 14:01:17.852899 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.852875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd" event={"ID":"723bc418-d780-4e89-b5ad-89dbe94ffda5","Type":"ContainerStarted","Data":"ada572ac9fbb9f3a109556cce691a4668c7517578a7a9f70040824845d60f99e"}
Jul 28 14:01:17.852998 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.852928    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd" event={"ID":"723bc418-d780-4e89-b5ad-89dbe94ffda5","Type":"ContainerStarted","Data":"92570dcac7d18e6a3e29c8055c595a2dbc9c689df5f73a5f9675d3301373970e"}
Jul 28 14:01:17.852998 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.852958    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd" event={"ID":"723bc418-d780-4e89-b5ad-89dbe94ffda5","Type":"ContainerStarted","Data":"dbbfeedf86c79ddb20c30192933ee1228ec0ccf009f989276a85fcafad27d86f"}
Jul 28 14:01:17.873303 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.873244    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-52cwh" podStartSLOduration=3.098156885 podStartE2EDuration="4.87322749s" totalImagesPullingTime="1.705793018s" totalInitContainerRuntime="69.277587ms" isStatefulPod=true podCreationTimestamp="2026-07-28 14:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:17.872162393 +0000 UTC m=+93.999553085" watchObservedRunningTime="2026-07-28 14:01:17.87322749 +0000 UTC m=+94.000618182"
Jul 28 14:01:17.891039 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:17.890981    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7w4jd" podStartSLOduration=2.206762254 podStartE2EDuration="4.89096635s" totalImagesPullingTime="2.684204096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:17.890231319 +0000 UTC m=+94.017622011" watchObservedRunningTime="2026-07-28 14:01:17.89096635 +0000 UTC m=+94.018357040"
Jul 28 14:01:18.621881 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.621853    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"]
Jul 28 14:01:18.625221 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.625204    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:18.627720 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.627698    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 14:01:18.627819 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.627699    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jrwp4\""
Jul 28 14:01:18.633300 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.633273    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"]
Jul 28 14:01:18.638067 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.638048    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7657b854fb-p5zb5"]
Jul 28 14:01:18.770169 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.770126    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhbqk\" (UniqueName: \"kubernetes.io/projected/42c4905e-b1cb-4a00-85c0-d68bf95ec142-kube-api-access-jhbqk\") pod \"monitoring-plugin-68b7fc9574-5mftw\" (UID: \"42c4905e-b1cb-4a00-85c0-d68bf95ec142\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:18.770350 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.770262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/42c4905e-b1cb-4a00-85c0-d68bf95ec142-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5mftw\" (UID: \"42c4905e-b1cb-4a00-85c0-d68bf95ec142\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:18.859757 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.859719    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c084083-aade-4fb9-854c-99fc8700fed6" containerID="1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde" exitCode=0
Jul 28 14:01:18.860247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.859806    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerDied","Data":"1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde"}
Jul 28 14:01:18.870881 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.870858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/42c4905e-b1cb-4a00-85c0-d68bf95ec142-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5mftw\" (UID: \"42c4905e-b1cb-4a00-85c0-d68bf95ec142\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:18.870991 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.870896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhbqk\" (UniqueName: \"kubernetes.io/projected/42c4905e-b1cb-4a00-85c0-d68bf95ec142-kube-api-access-jhbqk\") pod \"monitoring-plugin-68b7fc9574-5mftw\" (UID: \"42c4905e-b1cb-4a00-85c0-d68bf95ec142\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:18.873786 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.873724    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/42c4905e-b1cb-4a00-85c0-d68bf95ec142-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5mftw\" (UID: \"42c4905e-b1cb-4a00-85c0-d68bf95ec142\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:18.882827 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.882797    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhbqk\" (UniqueName: \"kubernetes.io/projected/42c4905e-b1cb-4a00-85c0-d68bf95ec142-kube-api-access-jhbqk\") pod \"monitoring-plugin-68b7fc9574-5mftw\" (UID: \"42c4905e-b1cb-4a00-85c0-d68bf95ec142\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:18.935267 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:18.935233    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:19.409720 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:19.409696    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"]
Jul 28 14:01:19.412589 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:01:19.412564    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c4905e_b1cb_4a00_85c0_d68bf95ec142.slice/crio-0d9e699bd419a5ef4bfaf74a7673454e1a927844d8c8edabf9d5cb8f175527d5 WatchSource:0}: Error finding container 0d9e699bd419a5ef4bfaf74a7673454e1a927844d8c8edabf9d5cb8f175527d5: Status 404 returned error can't find the container with id 0d9e699bd419a5ef4bfaf74a7673454e1a927844d8c8edabf9d5cb8f175527d5
Jul 28 14:01:19.867247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:19.867156    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h" event={"ID":"91b2b8f5-583d-4396-bbdb-0c96b3150ea3","Type":"ContainerStarted","Data":"fb7ff1153b6ba7dc2d2fc35541c742d996bc5f9d15046250eace02a454bebfc4"}
Jul 28 14:01:19.867247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:19.867240    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h" event={"ID":"91b2b8f5-583d-4396-bbdb-0c96b3150ea3","Type":"ContainerStarted","Data":"5cec5af0d08a4d79279f7630a26242be5373bdd567c63f5c0e6f6c56cd1b2529"}
Jul 28 14:01:19.867732 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:19.867280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h" event={"ID":"91b2b8f5-583d-4396-bbdb-0c96b3150ea3","Type":"ContainerStarted","Data":"d429d561a516c36f2fee3910d5c4a31a70a75ccd2a76415c1b570f9de04b1806"}
Jul 28 14:01:19.868593 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:19.868554    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw" event={"ID":"42c4905e-b1cb-4a00-85c0-d68bf95ec142","Type":"ContainerStarted","Data":"0d9e699bd419a5ef4bfaf74a7673454e1a927844d8c8edabf9d5cb8f175527d5"}
Jul 28 14:01:20.876968 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:20.876884    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerStarted","Data":"70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656"}
Jul 28 14:01:20.877375 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:20.876997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerStarted","Data":"139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123"}
Jul 28 14:01:20.877375 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:20.877059    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerStarted","Data":"67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678"}
Jul 28 14:01:20.877375 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:20.877088    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerStarted","Data":"b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca"}
Jul 28 14:01:20.877375 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:20.877112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerStarted","Data":"63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea"}
Jul 28 14:01:21.883092 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.882968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw" event={"ID":"42c4905e-b1cb-4a00-85c0-d68bf95ec142","Type":"ContainerStarted","Data":"de68ef7eb8fd55d37f0b86eae1222954f54e809ebdd75a89589f5b0ec650a28f"}
Jul 28 14:01:21.883092 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.883065    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:21.886970 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.886934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerStarted","Data":"a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44"}
Jul 28 14:01:21.888986 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.888955    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw"
Jul 28 14:01:21.890465 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.890443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h" event={"ID":"91b2b8f5-583d-4396-bbdb-0c96b3150ea3","Type":"ContainerStarted","Data":"924da4c006b34f95c265a60385cd88ef4d01801b50f2b7fbf4a24464c69ddd39"}
Jul 28 14:01:21.890595 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.890506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h" event={"ID":"91b2b8f5-583d-4396-bbdb-0c96b3150ea3","Type":"ContainerStarted","Data":"0313caa5c1284b3bc54952c4aabbb4db6f67963b0f2c7d844c274e2134d1ff71"}
Jul 28 14:01:21.890595 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.890556    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:21.890714 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.890595    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h" event={"ID":"91b2b8f5-583d-4396-bbdb-0c96b3150ea3","Type":"ContainerStarted","Data":"c167d774f273237ef108fc4f4b0d0e9ba182fcbd29753d82de70c5ed41e888de"}
Jul 28 14:01:21.900438 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.900398    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5mftw" podStartSLOduration=1.767725711 podStartE2EDuration="3.900384937s" totalImagesPullingTime="2.132659226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:21.899939594 +0000 UTC m=+98.027330288" watchObservedRunningTime="2026-07-28 14:01:21.900384937 +0000 UTC m=+98.027775608"
Jul 28 14:01:21.922632 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.922581    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h" podStartSLOduration=2.5424788659999997 podStartE2EDuration="5.922568018s" totalImagesPullingTime="3.380089152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:01:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:21.922407435 +0000 UTC m=+98.049798128" watchObservedRunningTime="2026-07-28 14:01:21.922568018 +0000 UTC m=+98.049958758"
Jul 28 14:01:21.950486 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:21.950437    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.7096107209999998 podStartE2EDuration="7.95041932s" totalImagesPullingTime="4.221748255s" totalInitContainerRuntime="19.060344ms" isStatefulPod=false podCreationTimestamp="2026-07-28 14:01:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:21.948205463 +0000 UTC m=+98.075596154" watchObservedRunningTime="2026-07-28 14:01:21.95041932 +0000 UTC m=+98.077810012"
Jul 28 14:01:22.723376 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.723344    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-595b6c7cb-s4fwb"]
Jul 28 14:01:22.726706 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.726686    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.730707 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.730683    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 14:01:22.731207 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.730809    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 14:01:22.731207 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.730842    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 14:01:22.731207 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.730944    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 14:01:22.731207 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.731084    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9vhmj\""
Jul 28 14:01:22.731470 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.731302    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 14:01:22.731617 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.731597    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 14:01:22.735341 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.735171    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 14:01:22.737100 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.737057    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 14:01:22.738263 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.737910    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-595b6c7cb-s4fwb"]
Jul 28 14:01:22.807839 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.807807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-trusted-ca-bundle\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.807938 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.807845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8twc\" (UniqueName: \"kubernetes.io/projected/fb7986ae-978b-4a9c-a302-d4e18479d9b3-kube-api-access-r8twc\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.807938 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.807874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-serving-cert\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.807938 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.807909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-oauth-config\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.808070 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.807950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-config\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.808070 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.807972    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-service-ca\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.808070 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.807994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-oauth-serving-cert\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.909359 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.909331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-config\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.909359 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.909359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-service-ca\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.909864 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.909635    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-oauth-serving-cert\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.909959 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.909922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-trusted-ca-bundle\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.910060 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.910006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-service-ca\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.910127 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.910040    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8twc\" (UniqueName: \"kubernetes.io/projected/fb7986ae-978b-4a9c-a302-d4e18479d9b3-kube-api-access-r8twc\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.910201 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.910161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-config\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.910277 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.910260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-oauth-serving-cert\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.910657 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.910633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-trusted-ca-bundle\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.910792 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.910778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-serving-cert\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.910928 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.910899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-oauth-config\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.913253 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.913230    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-oauth-config\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.913321 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.913301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-serving-cert\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:22.919401 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:22.919384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8twc\" (UniqueName: \"kubernetes.io/projected/fb7986ae-978b-4a9c-a302-d4e18479d9b3-kube-api-access-r8twc\") pod \"console-595b6c7cb-s4fwb\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") " pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:23.041288 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:23.041251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:23.176828 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:23.176797    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-595b6c7cb-s4fwb"]
Jul 28 14:01:23.180163 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:01:23.180138    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7986ae_978b_4a9c_a302_d4e18479d9b3.slice/crio-ba15bc4b24779ce8ee991a919180d4614b51ceba2b82c661e7a8f5bd4bd7747d WatchSource:0}: Error finding container ba15bc4b24779ce8ee991a919180d4614b51ceba2b82c661e7a8f5bd4bd7747d: Status 404 returned error can't find the container with id ba15bc4b24779ce8ee991a919180d4614b51ceba2b82c661e7a8f5bd4bd7747d
Jul 28 14:01:23.900797 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:23.900757    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595b6c7cb-s4fwb" event={"ID":"fb7986ae-978b-4a9c-a302-d4e18479d9b3","Type":"ContainerStarted","Data":"ba15bc4b24779ce8ee991a919180d4614b51ceba2b82c661e7a8f5bd4bd7747d"}
Jul 28 14:01:24.746516 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:24.746485    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-shngl"
Jul 28 14:01:26.915688 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:26.915646    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595b6c7cb-s4fwb" event={"ID":"fb7986ae-978b-4a9c-a302-d4e18479d9b3","Type":"ContainerStarted","Data":"d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf"}
Jul 28 14:01:26.936781 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:26.936725    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-595b6c7cb-s4fwb" podStartSLOduration=1.6934589519999999 podStartE2EDuration="4.936708992s" totalImagesPullingTime="3.24325004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:01:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:01:26.935176866 +0000 UTC m=+103.062567559" watchObservedRunningTime="2026-07-28 14:01:26.936708992 +0000 UTC m=+103.064099683"
Jul 28 14:01:27.901289 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:27.901261    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6f984bcdc7-qjj8h"
Jul 28 14:01:33.041749 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:33.041714    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:33.042235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:33.041820    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:33.046895 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:33.046871    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:33.947589 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:33.947562    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:01:34.949680 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:34.949648    2579 generic.go:356] "Generic (PLEG): container finished" podID="c32900e0-0735-40d4-946c-05ec3618add9" containerID="a4c10174585d8c510f195f0b3738acc9cbfa34f00d9b384b654398135a1479b6" exitCode=0
Jul 28 14:01:34.950147 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:34.949724    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-crr4w" event={"ID":"c32900e0-0735-40d4-946c-05ec3618add9","Type":"ContainerDied","Data":"a4c10174585d8c510f195f0b3738acc9cbfa34f00d9b384b654398135a1479b6"}
Jul 28 14:01:34.950297 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:34.950280    2579 scope.go:117] "RemoveContainer" containerID="a4c10174585d8c510f195f0b3738acc9cbfa34f00d9b384b654398135a1479b6"
Jul 28 14:01:35.956109 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:35.956070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-crr4w" event={"ID":"c32900e0-0735-40d4-946c-05ec3618add9","Type":"ContainerStarted","Data":"eaaecc03355fa5e6538cb59b8761858f67266953ca02ad738953e55c6b7b62d0"}
Jul 28 14:01:43.658441 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.658397    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5" podUID="7291bf28-e6bd-4c59-983e-78844bef303e" containerName="registry" containerID="cri-o://0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c" gracePeriod=30
Jul 28 14:01:43.741396 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.741366    2579 patch_prober.go:28] interesting pod/image-registry-7657b854fb-p5zb5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" start-of-body=
Jul 28 14:01:43.741540 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.741425    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5" podUID="7291bf28-e6bd-4c59-983e-78844bef303e" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused"
Jul 28 14:01:43.911814 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.911751    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:01:43.992947 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.992910    2579 generic.go:356] "Generic (PLEG): container finished" podID="7291bf28-e6bd-4c59-983e-78844bef303e" containerID="0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c" exitCode=0
Jul 28 14:01:43.993167 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.992976    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5"
Jul 28 14:01:43.993167 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.992985    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5" event={"ID":"7291bf28-e6bd-4c59-983e-78844bef303e","Type":"ContainerDied","Data":"0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c"}
Jul 28 14:01:43.993167 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.993084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7657b854fb-p5zb5" event={"ID":"7291bf28-e6bd-4c59-983e-78844bef303e","Type":"ContainerDied","Data":"78583ff66cda9ebc6bb3043afe982daa11188f48ea874e7db717e67916b173ee"}
Jul 28 14:01:43.993167 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.993106    2579 scope.go:117] "RemoveContainer" containerID="0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c"
Jul 28 14:01:43.993825 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.993807    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws57g\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-kube-api-access-ws57g\") pod \"7291bf28-e6bd-4c59-983e-78844bef303e\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") "
Jul 28 14:01:43.993934 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.993849    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-trusted-ca\") pod \"7291bf28-e6bd-4c59-983e-78844bef303e\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") "
Jul 28 14:01:43.994087 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.994002    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7291bf28-e6bd-4c59-983e-78844bef303e-ca-trust-extracted\") pod \"7291bf28-e6bd-4c59-983e-78844bef303e\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") "
Jul 28 14:01:43.994148 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.994085    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-image-registry-private-configuration\") pod \"7291bf28-e6bd-4c59-983e-78844bef303e\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") "
Jul 28 14:01:43.994148 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.994116    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-registry-certificates\") pod \"7291bf28-e6bd-4c59-983e-78844bef303e\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") "
Jul 28 14:01:43.994293 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.994172    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-installation-pull-secrets\") pod \"7291bf28-e6bd-4c59-983e-78844bef303e\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") "
Jul 28 14:01:43.994293 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.994203    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") pod \"7291bf28-e6bd-4c59-983e-78844bef303e\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") "
Jul 28 14:01:43.994293 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.994224    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-bound-sa-token\") pod \"7291bf28-e6bd-4c59-983e-78844bef303e\" (UID: \"7291bf28-e6bd-4c59-983e-78844bef303e\") "
Jul 28 14:01:43.994439 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.994299    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7291bf28-e6bd-4c59-983e-78844bef303e" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:01:43.994505 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.994483    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7291bf28-e6bd-4c59-983e-78844bef303e" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:01:43.994815 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.994796    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-registry-certificates\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:01:43.995168 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.995150    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7291bf28-e6bd-4c59-983e-78844bef303e-trusted-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:01:43.997359 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.997334    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-kube-api-access-ws57g" (OuterVolumeSpecName: "kube-api-access-ws57g") pod "7291bf28-e6bd-4c59-983e-78844bef303e" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e"). InnerVolumeSpecName "kube-api-access-ws57g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:01:43.997531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.997381    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7291bf28-e6bd-4c59-983e-78844bef303e" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:01:43.997531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.997451    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7291bf28-e6bd-4c59-983e-78844bef303e" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:01:43.997670 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.997576    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7291bf28-e6bd-4c59-983e-78844bef303e" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:01:43.998371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:43.998346    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7291bf28-e6bd-4c59-983e-78844bef303e" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:01:44.003370 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.003340    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7291bf28-e6bd-4c59-983e-78844bef303e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7291bf28-e6bd-4c59-983e-78844bef303e" (UID: "7291bf28-e6bd-4c59-983e-78844bef303e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:01:44.013296 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.013277    2579 scope.go:117] "RemoveContainer" containerID="0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c"
Jul 28 14:01:44.013591 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:01:44.013572    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c\": container with ID starting with 0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c not found: ID does not exist" containerID="0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c"
Jul 28 14:01:44.013640 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.013598    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c"} err="failed to get container status \"0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c\": rpc error: code = NotFound desc = could not find container \"0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c\": container with ID starting with 0df1a4d2ec20495e278474fb2f0821f17b8cf3435faa582c4b7211dd6c19240c not found: ID does not exist"
Jul 28 14:01:44.096259 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.096221    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7291bf28-e6bd-4c59-983e-78844bef303e-ca-trust-extracted\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:01:44.096259 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.096252    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-image-registry-private-configuration\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:01:44.096259 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.096262    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7291bf28-e6bd-4c59-983e-78844bef303e-installation-pull-secrets\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:01:44.096481 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.096273    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-registry-tls\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:01:44.096481 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.096281    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-bound-sa-token\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:01:44.096481 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.096290    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws57g\" (UniqueName: \"kubernetes.io/projected/7291bf28-e6bd-4c59-983e-78844bef303e-kube-api-access-ws57g\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:01:44.318781 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.318746    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7657b854fb-p5zb5"]
Jul 28 14:01:44.323338 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.323315    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7657b854fb-p5zb5"]
Jul 28 14:01:44.341126 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:01:44.341095    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7291bf28-e6bd-4c59-983e-78844bef303e" path="/var/lib/kubelet/pods/7291bf28-e6bd-4c59-983e-78844bef303e/volumes"
Jul 28 14:02:00.056003 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:00.055967    2579 generic.go:356] "Generic (PLEG): container finished" podID="b4915355-1200-44ec-a2ef-2a8ace1c6348" containerID="f8f0998a2ff1049a4964e9f1d1e2a37ba471715af96f1a21bd0490ccaa358cdd" exitCode=0
Jul 28 14:02:00.056529 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:00.056046    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b" event={"ID":"b4915355-1200-44ec-a2ef-2a8ace1c6348","Type":"ContainerDied","Data":"f8f0998a2ff1049a4964e9f1d1e2a37ba471715af96f1a21bd0490ccaa358cdd"}
Jul 28 14:02:00.056529 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:00.056445    2579 scope.go:117] "RemoveContainer" containerID="f8f0998a2ff1049a4964e9f1d1e2a37ba471715af96f1a21bd0490ccaa358cdd"
Jul 28 14:02:00.057753 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:00.057731    2579 generic.go:356] "Generic (PLEG): container finished" podID="85a8fcc2-28d8-4534-908a-c0ea8fdf51ba" containerID="93ff8f34280e8ed99d3506d03efe50256fb100dbc631d883bbe3262fa2e47f9b" exitCode=0
Jul 28 14:02:00.057846 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:00.057796    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw" event={"ID":"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba","Type":"ContainerDied","Data":"93ff8f34280e8ed99d3506d03efe50256fb100dbc631d883bbe3262fa2e47f9b"}
Jul 28 14:02:00.058108 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:00.058094    2579 scope.go:117] "RemoveContainer" containerID="93ff8f34280e8ed99d3506d03efe50256fb100dbc631d883bbe3262fa2e47f9b"
Jul 28 14:02:01.064137 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:01.064095    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-72w7b" event={"ID":"b4915355-1200-44ec-a2ef-2a8ace1c6348","Type":"ContainerStarted","Data":"d2296562b6dd10d857ad26cbe3156d46482c11f8e54da93916bfcfe065c4790b"}
Jul 28 14:02:01.065867 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:01.065844    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw" event={"ID":"85a8fcc2-28d8-4534-908a-c0ea8fdf51ba","Type":"ContainerStarted","Data":"ef400ba459c52a39b3c7a95ddcecff3ef68f050f6c0616754fd7a7fc23ab3727"}
Jul 28 14:02:34.340853 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:34.340814    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:02:34.341435 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:34.341150    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="alertmanager" containerID="cri-o://63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea" gracePeriod=120
Jul 28 14:02:34.341435 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:34.341192    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy-metric" containerID="cri-o://70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656" gracePeriod=120
Jul 28 14:02:34.341435 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:34.341239    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="prom-label-proxy" containerID="cri-o://a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44" gracePeriod=120
Jul 28 14:02:34.341435 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:34.341267    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="config-reloader" containerID="cri-o://b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca" gracePeriod=120
Jul 28 14:02:34.341435 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:34.341295    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy" containerID="cri-o://139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123" gracePeriod=120
Jul 28 14:02:34.341435 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:34.341239    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy-web" containerID="cri-o://67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678" gracePeriod=120
Jul 28 14:02:35.211224 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.211151    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c084083-aade-4fb9-854c-99fc8700fed6" containerID="a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44" exitCode=0
Jul 28 14:02:35.211224 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.211177    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c084083-aade-4fb9-854c-99fc8700fed6" containerID="139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123" exitCode=0
Jul 28 14:02:35.211224 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.211183    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c084083-aade-4fb9-854c-99fc8700fed6" containerID="b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca" exitCode=0
Jul 28 14:02:35.211224 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.211191    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c084083-aade-4fb9-854c-99fc8700fed6" containerID="63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea" exitCode=0
Jul 28 14:02:35.211458 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.211229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerDied","Data":"a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44"}
Jul 28 14:02:35.211458 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.211313    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerDied","Data":"139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123"}
Jul 28 14:02:35.211458 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.211339    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerDied","Data":"b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca"}
Jul 28 14:02:35.211458 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.211363    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerDied","Data":"63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea"}
Jul 28 14:02:35.602446 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.602424    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:35.736471 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736440    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-main-tls\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.736642 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736492    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-cluster-tls-config\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.736702 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736650    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-main-db\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.736774 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736714    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-trusted-ca-bundle\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.736774 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736745    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.736872 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736807    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-config-volume\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.736872 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736834    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.736872 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736861    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-web-config\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.737049 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-config-out\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.737049 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736921    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-web\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.737049 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736953    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-tls-assets\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.737049 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-metrics-client-ca\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.737049 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.736997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:02:35.737300 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.737056    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52jzv\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-kube-api-access-52jzv\") pod \"0c084083-aade-4fb9-854c-99fc8700fed6\" (UID: \"0c084083-aade-4fb9-854c-99fc8700fed6\") "
Jul 28 14:02:35.737357 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.737334    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:02:35.737420 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.737354    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-main-db\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.738077 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.737974    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:02:35.739866 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.739843    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:02:35.740478 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.740428    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-kube-api-access-52jzv" (OuterVolumeSpecName: "kube-api-access-52jzv") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "kube-api-access-52jzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:02:35.740478 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.740467    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-config-out" (OuterVolumeSpecName: "config-out") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:02:35.740635 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.740534    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:02:35.740790 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.740763    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-config-volume" (OuterVolumeSpecName: "config-volume") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:02:35.740790 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.740773    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:02:35.740967 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.740786    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:02:35.741718 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.741701    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:02:35.744235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.744211    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:02:35.751979 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.751961    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-web-config" (OuterVolumeSpecName: "web-config") pod "0c084083-aade-4fb9-854c-99fc8700fed6" (UID: "0c084083-aade-4fb9-854c-99fc8700fed6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:02:35.838541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838472    2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-config-volume\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838507    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838517    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-web-config\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838525    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c084083-aade-4fb9-854c-99fc8700fed6-config-out\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838534    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838542    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-tls-assets\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838551    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-metrics-client-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838820 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838559    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52jzv\" (UniqueName: \"kubernetes.io/projected/0c084083-aade-4fb9-854c-99fc8700fed6-kube-api-access-52jzv\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838820 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838569    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-main-tls\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838820 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838577    2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-cluster-tls-config\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838820 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838585    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c084083-aade-4fb9-854c-99fc8700fed6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:35.838820 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:35.838593    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c084083-aade-4fb9-854c-99fc8700fed6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:02:36.218447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.218367    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c084083-aade-4fb9-854c-99fc8700fed6" containerID="70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656" exitCode=0
Jul 28 14:02:36.218447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.218398    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c084083-aade-4fb9-854c-99fc8700fed6" containerID="67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678" exitCode=0
Jul 28 14:02:36.218447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.218438    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerDied","Data":"70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656"}
Jul 28 14:02:36.218647 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.218470    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.218647 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.218514    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerDied","Data":"67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678"}
Jul 28 14:02:36.218647 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.218556    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c084083-aade-4fb9-854c-99fc8700fed6","Type":"ContainerDied","Data":"23bb16ba44825dc76318398ba7fbf909c3c24623a26a6c07348c5ae18864a7e3"}
Jul 28 14:02:36.218647 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.218586    2579 scope.go:117] "RemoveContainer" containerID="a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44"
Jul 28 14:02:36.228099 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.228080    2579 scope.go:117] "RemoveContainer" containerID="70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656"
Jul 28 14:02:36.235617 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.235597    2579 scope.go:117] "RemoveContainer" containerID="139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123"
Jul 28 14:02:36.242409 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.242392    2579 scope.go:117] "RemoveContainer" containerID="67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678"
Jul 28 14:02:36.248914 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.248891    2579 scope.go:117] "RemoveContainer" containerID="b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca"
Jul 28 14:02:36.252514 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.252456    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:02:36.256678 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.256660    2579 scope.go:117] "RemoveContainer" containerID="63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea"
Jul 28 14:02:36.260125 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.260107    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:02:36.264178 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.264161    2579 scope.go:117] "RemoveContainer" containerID="1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde"
Jul 28 14:02:36.270671 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.270657    2579 scope.go:117] "RemoveContainer" containerID="a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44"
Jul 28 14:02:36.270903 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:02:36.270884    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44\": container with ID starting with a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44 not found: ID does not exist" containerID="a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44"
Jul 28 14:02:36.270960 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.270913    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44"} err="failed to get container status \"a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44\": rpc error: code = NotFound desc = could not find container \"a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44\": container with ID starting with a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44 not found: ID does not exist"
Jul 28 14:02:36.270960 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.270929    2579 scope.go:117] "RemoveContainer" containerID="70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656"
Jul 28 14:02:36.271185 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:02:36.271167    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656\": container with ID starting with 70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656 not found: ID does not exist" containerID="70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656"
Jul 28 14:02:36.271229 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.271192    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656"} err="failed to get container status \"70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656\": rpc error: code = NotFound desc = could not find container \"70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656\": container with ID starting with 70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656 not found: ID does not exist"
Jul 28 14:02:36.271229 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.271208    2579 scope.go:117] "RemoveContainer" containerID="139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123"
Jul 28 14:02:36.271402 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:02:36.271388    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123\": container with ID starting with 139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123 not found: ID does not exist" containerID="139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123"
Jul 28 14:02:36.271446 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.271406    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123"} err="failed to get container status \"139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123\": rpc error: code = NotFound desc = could not find container \"139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123\": container with ID starting with 139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123 not found: ID does not exist"
Jul 28 14:02:36.271446 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.271419    2579 scope.go:117] "RemoveContainer" containerID="67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678"
Jul 28 14:02:36.271659 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:02:36.271643    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678\": container with ID starting with 67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678 not found: ID does not exist" containerID="67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678"
Jul 28 14:02:36.271703 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.271663    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678"} err="failed to get container status \"67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678\": rpc error: code = NotFound desc = could not find container \"67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678\": container with ID starting with 67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678 not found: ID does not exist"
Jul 28 14:02:36.271703 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.271676    2579 scope.go:117] "RemoveContainer" containerID="b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca"
Jul 28 14:02:36.271869 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:02:36.271852    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca\": container with ID starting with b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca not found: ID does not exist" containerID="b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca"
Jul 28 14:02:36.271929 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.271877    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca"} err="failed to get container status \"b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca\": rpc error: code = NotFound desc = could not find container \"b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca\": container with ID starting with b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca not found: ID does not exist"
Jul 28 14:02:36.271929 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.271898    2579 scope.go:117] "RemoveContainer" containerID="63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea"
Jul 28 14:02:36.272147 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:02:36.272128    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea\": container with ID starting with 63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea not found: ID does not exist" containerID="63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea"
Jul 28 14:02:36.272197 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.272152    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea"} err="failed to get container status \"63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea\": rpc error: code = NotFound desc = could not find container \"63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea\": container with ID starting with 63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea not found: ID does not exist"
Jul 28 14:02:36.272197 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.272166    2579 scope.go:117] "RemoveContainer" containerID="1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde"
Jul 28 14:02:36.272408 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:02:36.272392    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde\": container with ID starting with 1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde not found: ID does not exist" containerID="1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde"
Jul 28 14:02:36.272450 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.272412    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde"} err="failed to get container status \"1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde\": rpc error: code = NotFound desc = could not find container \"1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde\": container with ID starting with 1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde not found: ID does not exist"
Jul 28 14:02:36.272450 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.272424    2579 scope.go:117] "RemoveContainer" containerID="a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44"
Jul 28 14:02:36.272634 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.272619    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44"} err="failed to get container status \"a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44\": rpc error: code = NotFound desc = could not find container \"a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44\": container with ID starting with a975e715ff43206cec78ffe800cf4093da7e9572c66440cce0fcad532aeb6c44 not found: ID does not exist"
Jul 28 14:02:36.272677 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.272635    2579 scope.go:117] "RemoveContainer" containerID="70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656"
Jul 28 14:02:36.272838 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.272820    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656"} err="failed to get container status \"70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656\": rpc error: code = NotFound desc = could not find container \"70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656\": container with ID starting with 70fdb2113a8f0a27dd3ac66a3bfa29c61e66dbd009705b8eef23430fd34dc656 not found: ID does not exist"
Jul 28 14:02:36.272877 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.272840    2579 scope.go:117] "RemoveContainer" containerID="139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123"
Jul 28 14:02:36.273081 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.273063    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123"} err="failed to get container status \"139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123\": rpc error: code = NotFound desc = could not find container \"139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123\": container with ID starting with 139e05c065fc23667c102ee4dacad9df7e8018ccb68fc45cbbfea3e5b82ef123 not found: ID does not exist"
Jul 28 14:02:36.273128 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.273084    2579 scope.go:117] "RemoveContainer" containerID="67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678"
Jul 28 14:02:36.273287 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.273270    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678"} err="failed to get container status \"67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678\": rpc error: code = NotFound desc = could not find container \"67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678\": container with ID starting with 67b218d34f2380e79418f077b1ea63c585670efd7ef9e021478cefcabfc60678 not found: ID does not exist"
Jul 28 14:02:36.273358 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.273290    2579 scope.go:117] "RemoveContainer" containerID="b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca"
Jul 28 14:02:36.273534 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.273517    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca"} err="failed to get container status \"b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca\": rpc error: code = NotFound desc = could not find container \"b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca\": container with ID starting with b9b79d83def54290f5dffd7fad5937ef2e8d2faf22e63f82fa896abfca4ec4ca not found: ID does not exist"
Jul 28 14:02:36.273578 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.273534    2579 scope.go:117] "RemoveContainer" containerID="63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea"
Jul 28 14:02:36.273754 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.273734    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea"} err="failed to get container status \"63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea\": rpc error: code = NotFound desc = could not find container \"63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea\": container with ID starting with 63656f4582c92245c11ca03806272a8039b1f7328403c5f3e5e6aa3855a6e9ea not found: ID does not exist"
Jul 28 14:02:36.273825 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.273755    2579 scope.go:117] "RemoveContainer" containerID="1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde"
Jul 28 14:02:36.273970 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.273946    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde"} err="failed to get container status \"1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde\": rpc error: code = NotFound desc = could not find container \"1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde\": container with ID starting with 1880f9bd50a458c20d2bacceaef08a9ce75095054d94f3af3e1763357aec1dde not found: ID does not exist"
Jul 28 14:02:36.299691 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.299663    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:02:36.299980 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.299966    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="init-config-reloader"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.299982    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="init-config-reloader"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.299995    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="alertmanager"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300002    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="alertmanager"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300008    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300026    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300033    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy-metric"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300038    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy-metric"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300046    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="config-reloader"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300050    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="config-reloader"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300057    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy-web"
Jul 28 14:02:36.300066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300061    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy-web"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300076    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7291bf28-e6bd-4c59-983e-78844bef303e" containerName="registry"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300107    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7291bf28-e6bd-4c59-983e-78844bef303e" containerName="registry"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300113    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="prom-label-proxy"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300118    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="prom-label-proxy"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300186    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="alertmanager"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300193    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="config-reloader"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300199    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="prom-label-proxy"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300207    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7291bf28-e6bd-4c59-983e-78844bef303e" containerName="registry"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300213    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy-web"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300218    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy"
Jul 28 14:02:36.300419 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.300225    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" containerName="kube-rbac-proxy-metric"
Jul 28 14:02:36.305176 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.305162    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.308921 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.308901    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 14:02:36.309002 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.308932    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 14:02:36.309365 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.309349    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 14:02:36.309726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.309712    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 14:02:36.309804 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.309790    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 14:02:36.310252 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.310239    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 14:02:36.310742 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.310721    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 14:02:36.310887 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.310862    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 14:02:36.312788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.312771    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dl28v\""
Jul 28 14:02:36.318632 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.318614    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 14:02:36.330813 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.330784    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:02:36.342061 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.342033    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c084083-aade-4fb9-854c-99fc8700fed6" path="/var/lib/kubelet/pods/0c084083-aade-4fb9-854c-99fc8700fed6/volumes"
Jul 28 14:02:36.443593 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.443561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.443593 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.443595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7524694e-0ec3-469f-af9c-bdfb7158eb3a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.443765 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.443682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7524694e-0ec3-469f-af9c-bdfb7158eb3a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.443814 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.443798    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7524694e-0ec3-469f-af9c-bdfb7158eb3a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.443894 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.443880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7524694e-0ec3-469f-af9c-bdfb7158eb3a-config-out\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.443953 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.443900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjjrf\" (UniqueName: \"kubernetes.io/projected/7524694e-0ec3-469f-af9c-bdfb7158eb3a-kube-api-access-mjjrf\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.443953 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.443934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.444089 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.443959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.444089 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.443981    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-web-config\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.444089 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.444054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7524694e-0ec3-469f-af9c-bdfb7158eb3a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.444089 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.444088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.444232 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.444116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.444232 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.444145    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-config-volume\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.544875 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.544847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-config-volume\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545095 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.544888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545095 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.544913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7524694e-0ec3-469f-af9c-bdfb7158eb3a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545095 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7524694e-0ec3-469f-af9c-bdfb7158eb3a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545274 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545096    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7524694e-0ec3-469f-af9c-bdfb7158eb3a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545274 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545149    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7524694e-0ec3-469f-af9c-bdfb7158eb3a-config-out\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545274 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjjrf\" (UniqueName: \"kubernetes.io/projected/7524694e-0ec3-469f-af9c-bdfb7158eb3a-kube-api-access-mjjrf\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545274 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545274 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545274 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-web-config\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545558 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7524694e-0ec3-469f-af9c-bdfb7158eb3a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545558 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545558 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545353    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7524694e-0ec3-469f-af9c-bdfb7158eb3a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545558 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.545978 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.545944    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7524694e-0ec3-469f-af9c-bdfb7158eb3a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.546730 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.546413    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7524694e-0ec3-469f-af9c-bdfb7158eb3a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.548171 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.548051    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7524694e-0ec3-469f-af9c-bdfb7158eb3a-config-out\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.548468 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.548439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7524694e-0ec3-469f-af9c-bdfb7158eb3a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.548567 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.548514    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.548654 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.548633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-config-volume\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.548806 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.548785    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.548895 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.548876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.548930 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.548876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-web-config\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.549306 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.549286    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.550266 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.550249    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7524694e-0ec3-469f-af9c-bdfb7158eb3a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.555544 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.555524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjjrf\" (UniqueName: \"kubernetes.io/projected/7524694e-0ec3-469f-af9c-bdfb7158eb3a-kube-api-access-mjjrf\") pod \"alertmanager-main-0\" (UID: \"7524694e-0ec3-469f-af9c-bdfb7158eb3a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.615333 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.615297    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:02:36.767468 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:36.767435    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:02:36.770751 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:02:36.770724    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7524694e_0ec3_469f_af9c_bdfb7158eb3a.slice/crio-1d19503ed1054802fd310f40daa3cf0d0e8a3fff3ee8be83297d1d3f45cca5d8 WatchSource:0}: Error finding container 1d19503ed1054802fd310f40daa3cf0d0e8a3fff3ee8be83297d1d3f45cca5d8: Status 404 returned error can't find the container with id 1d19503ed1054802fd310f40daa3cf0d0e8a3fff3ee8be83297d1d3f45cca5d8
Jul 28 14:02:37.224358 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:37.224274    2579 generic.go:356] "Generic (PLEG): container finished" podID="7524694e-0ec3-469f-af9c-bdfb7158eb3a" containerID="8029163e679b3abe3ff7a7a1fbe7f7a80c4c8fb1f98d208114a11caf68f0101e" exitCode=0
Jul 28 14:02:37.224358 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:37.224319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7524694e-0ec3-469f-af9c-bdfb7158eb3a","Type":"ContainerDied","Data":"8029163e679b3abe3ff7a7a1fbe7f7a80c4c8fb1f98d208114a11caf68f0101e"}
Jul 28 14:02:37.224557 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:37.224436    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7524694e-0ec3-469f-af9c-bdfb7158eb3a","Type":"ContainerStarted","Data":"1d19503ed1054802fd310f40daa3cf0d0e8a3fff3ee8be83297d1d3f45cca5d8"}
Jul 28 14:02:38.232249 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.232217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7524694e-0ec3-469f-af9c-bdfb7158eb3a","Type":"ContainerStarted","Data":"60d463a2be48b3a61ac7b9438e5f22809a0f1979b6f8ab3c8f9c9108f72e0063"}
Jul 28 14:02:38.232750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.232325    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7524694e-0ec3-469f-af9c-bdfb7158eb3a","Type":"ContainerStarted","Data":"e4858ef09271eed2f0d82840409cedcd62d55e0182028830a02de0fd58e65608"}
Jul 28 14:02:38.232750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.232381    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7524694e-0ec3-469f-af9c-bdfb7158eb3a","Type":"ContainerStarted","Data":"3046b69a5431b833b7a2191179fc1b011e7065b82d3af701100d4d03cb34b4c3"}
Jul 28 14:02:38.232750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.232420    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7524694e-0ec3-469f-af9c-bdfb7158eb3a","Type":"ContainerStarted","Data":"94ad01ce2ac2a44989ba086287d95271164f59f7375fa999a3cd3d158e9418b7"}
Jul 28 14:02:38.232750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.232466    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7524694e-0ec3-469f-af9c-bdfb7158eb3a","Type":"ContainerStarted","Data":"8ebda6e0ba62ab53de515b56a940e85263f99cfdb67a726e1cc7439880cace45"}
Jul 28 14:02:38.232750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.232503    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7524694e-0ec3-469f-af9c-bdfb7158eb3a","Type":"ContainerStarted","Data":"2ed05ab9f49f9bc8e36859570869dcb4f4796f39c90c1895f64a96c37ec5fc8a"}
Jul 28 14:02:38.270507 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.270460    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.250711956 podStartE2EDuration="2.270445246s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.73329ms" isStatefulPod=false podCreationTimestamp="2026-07-28 14:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:02:38.270419021 +0000 UTC m=+174.397809711" watchObservedRunningTime="2026-07-28 14:02:38.270445246 +0000 UTC m=+174.397835938"
Jul 28 14:02:38.482487 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.482402    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-64775789b7-6rkb8"]
Jul 28 14:02:38.486107 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.486080    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.488723 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.488694    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-x2n64\""
Jul 28 14:02:38.488824 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.488793    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 14:02:38.489760 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.489739    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 14:02:38.490011 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.489995    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 14:02:38.490100 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.490090    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 14:02:38.490316 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.490299    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 14:02:38.494637 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.494611    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 14:02:38.515185 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.515161    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64775789b7-6rkb8"]
Jul 28 14:02:38.563549 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.563524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-telemeter-client-tls\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.563670 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.563552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2nt2\" (UniqueName: \"kubernetes.io/projected/ba905d8d-911d-4b94-ad05-50927a13c5af-kube-api-access-w2nt2\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.563670 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.563570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-federate-client-tls\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.563670 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.563640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba905d8d-911d-4b94-ad05-50927a13c5af-metrics-client-ca\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.563816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.563679    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba905d8d-911d-4b94-ad05-50927a13c5af-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.563816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.563704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba905d8d-911d-4b94-ad05-50927a13c5af-serving-certs-ca-bundle\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.563816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.563725    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.563816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.563777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-secret-telemeter-client\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.664852 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.664826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba905d8d-911d-4b94-ad05-50927a13c5af-metrics-client-ca\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.664980 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.664860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba905d8d-911d-4b94-ad05-50927a13c5af-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.664980 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.664881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba905d8d-911d-4b94-ad05-50927a13c5af-serving-certs-ca-bundle\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.664980 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.664899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.664980 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.664925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-secret-telemeter-client\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.664980 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.664950    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-telemeter-client-tls\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.665263 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.665065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2nt2\" (UniqueName: \"kubernetes.io/projected/ba905d8d-911d-4b94-ad05-50927a13c5af-kube-api-access-w2nt2\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.665263 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.665098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-federate-client-tls\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.665906 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.665670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba905d8d-911d-4b94-ad05-50927a13c5af-metrics-client-ca\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.665906 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.665719    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba905d8d-911d-4b94-ad05-50927a13c5af-serving-certs-ca-bundle\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.666080 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.666048    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba905d8d-911d-4b94-ad05-50927a13c5af-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.667680 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.667654    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-telemeter-client-tls\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.667766 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.667744    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-secret-telemeter-client\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.667886 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.667865    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.667944 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.667884    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ba905d8d-911d-4b94-ad05-50927a13c5af-federate-client-tls\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.675564 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.675545    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2nt2\" (UniqueName: \"kubernetes.io/projected/ba905d8d-911d-4b94-ad05-50927a13c5af-kube-api-access-w2nt2\") pod \"telemeter-client-64775789b7-6rkb8\" (UID: \"ba905d8d-911d-4b94-ad05-50927a13c5af\") " pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.795689 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.795660    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8"
Jul 28 14:02:38.942290 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:38.942264    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64775789b7-6rkb8"]
Jul 28 14:02:38.944777 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:02:38.944749    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba905d8d_911d_4b94_ad05_50927a13c5af.slice/crio-2d952cf27d30beb70a8c33c38aec1d42318cc88f7d393f7c5cf24b5e15684406 WatchSource:0}: Error finding container 2d952cf27d30beb70a8c33c38aec1d42318cc88f7d393f7c5cf24b5e15684406: Status 404 returned error can't find the container with id 2d952cf27d30beb70a8c33c38aec1d42318cc88f7d393f7c5cf24b5e15684406
Jul 28 14:02:39.238201 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:39.238112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8" event={"ID":"ba905d8d-911d-4b94-ad05-50927a13c5af","Type":"ContainerStarted","Data":"2d952cf27d30beb70a8c33c38aec1d42318cc88f7d393f7c5cf24b5e15684406"}
Jul 28 14:02:41.250675 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:41.250585    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8" event={"ID":"ba905d8d-911d-4b94-ad05-50927a13c5af","Type":"ContainerStarted","Data":"f3818271dd90be6805106e178552e0b23ba3c1e7159a712c42fc9f4ac4fb92e7"}
Jul 28 14:02:41.251181 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:41.250692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8" event={"ID":"ba905d8d-911d-4b94-ad05-50927a13c5af","Type":"ContainerStarted","Data":"c34ebb5b5a21cdbf561d7ef1adbef4822af8375020e1a5f2fe5f21dfd57b9f41"}
Jul 28 14:02:41.251181 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:41.250720    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8" event={"ID":"ba905d8d-911d-4b94-ad05-50927a13c5af","Type":"ContainerStarted","Data":"df5f5289c400676275bb7f5e63aab8f34a5ed94712dc0c3536830319f0216ee5"}
Jul 28 14:02:41.304313 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:41.304263    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-64775789b7-6rkb8" podStartSLOduration=1.381900587 podStartE2EDuration="3.304250264s" totalImagesPullingTime="1.922349677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:02:41.304052016 +0000 UTC m=+177.431442713" watchObservedRunningTime="2026-07-28 14:02:41.304250264 +0000 UTC m=+177.431640954"
Jul 28 14:02:42.392426 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.392398    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8669f8fc6b-jxq45"]
Jul 28 14:02:42.395836 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.395813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.414371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.414346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8669f8fc6b-jxq45"]
Jul 28 14:02:42.499302 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.499272    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b7zf\" (UniqueName: \"kubernetes.io/projected/415f75a8-97f5-4bf3-8170-b333490160f4-kube-api-access-7b7zf\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.499457 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.499311    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-serving-cert\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.499457 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.499330    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-oauth-config\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.499457 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.499374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-console-config\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.499457 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.499422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-trusted-ca-bundle\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.499596 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.499480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-oauth-serving-cert\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.499596 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.499528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-service-ca\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.600742 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.600706    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-oauth-serving-cert\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.600918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.600749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-service-ca\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.600918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.600782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b7zf\" (UniqueName: \"kubernetes.io/projected/415f75a8-97f5-4bf3-8170-b333490160f4-kube-api-access-7b7zf\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.600918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.600814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-serving-cert\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.600918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.600837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-oauth-config\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.600918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.600858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-console-config\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.600918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.600894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-trusted-ca-bundle\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.601695 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.601574    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-oauth-serving-cert\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.601695 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.601599    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-service-ca\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.601695 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.601603    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-console-config\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.601987 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.601963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-trusted-ca-bundle\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.603442 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.603421    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-serving-cert\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.603545 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.603527    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-oauth-config\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.611245 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.611227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b7zf\" (UniqueName: \"kubernetes.io/projected/415f75a8-97f5-4bf3-8170-b333490160f4-kube-api-access-7b7zf\") pod \"console-8669f8fc6b-jxq45\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") " pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.706655 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.706574    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:42.840367 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:42.840342    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8669f8fc6b-jxq45"]
Jul 28 14:02:42.842632 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:02:42.842606    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod415f75a8_97f5_4bf3_8170_b333490160f4.slice/crio-89ea75a566f11bdc483e01946cd22eddbcb06adecdc9d00b54c6fdb927e723a0 WatchSource:0}: Error finding container 89ea75a566f11bdc483e01946cd22eddbcb06adecdc9d00b54c6fdb927e723a0: Status 404 returned error can't find the container with id 89ea75a566f11bdc483e01946cd22eddbcb06adecdc9d00b54c6fdb927e723a0
Jul 28 14:02:43.263404 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:43.263369    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669f8fc6b-jxq45" event={"ID":"415f75a8-97f5-4bf3-8170-b333490160f4","Type":"ContainerStarted","Data":"8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a"}
Jul 28 14:02:43.263572 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:43.263424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669f8fc6b-jxq45" event={"ID":"415f75a8-97f5-4bf3-8170-b333490160f4","Type":"ContainerStarted","Data":"89ea75a566f11bdc483e01946cd22eddbcb06adecdc9d00b54c6fdb927e723a0"}
Jul 28 14:02:43.286273 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:43.286198    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8669f8fc6b-jxq45" podStartSLOduration=1.286177033 podStartE2EDuration="1.286177033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:02:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:02:43.285865096 +0000 UTC m=+179.413255787" watchObservedRunningTime="2026-07-28 14:02:43.286177033 +0000 UTC m=+179.413567723"
Jul 28 14:02:52.707114 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:52.707078    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:52.707486 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:52.707141    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:52.711812 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:52.711791    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:53.305507 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:53.305478    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:02:53.354404 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:02:53.354349    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-595b6c7cb-s4fwb"]
Jul 28 14:03:18.381550 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.381485    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-595b6c7cb-s4fwb" podUID="fb7986ae-978b-4a9c-a302-d4e18479d9b3" containerName="console" containerID="cri-o://d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf" gracePeriod=15
Jul 28 14:03:18.623812 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.623784    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:03:18.719436 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.719361    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-serving-cert\") pod \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") "
Jul 28 14:03:18.719570 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.719434    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-config\") pod \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") "
Jul 28 14:03:18.719570 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.719474    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-oauth-config\") pod \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") "
Jul 28 14:03:18.719570 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.719492    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-service-ca\") pod \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") "
Jul 28 14:03:18.719570 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.719521    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-trusted-ca-bundle\") pod \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") "
Jul 28 14:03:18.719767 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.719586    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8twc\" (UniqueName: \"kubernetes.io/projected/fb7986ae-978b-4a9c-a302-d4e18479d9b3-kube-api-access-r8twc\") pod \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") "
Jul 28 14:03:18.719767 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.719621    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-oauth-serving-cert\") pod \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\" (UID: \"fb7986ae-978b-4a9c-a302-d4e18479d9b3\") "
Jul 28 14:03:18.719920 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.719893    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-config" (OuterVolumeSpecName: "console-config") pod "fb7986ae-978b-4a9c-a302-d4e18479d9b3" (UID: "fb7986ae-978b-4a9c-a302-d4e18479d9b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:03:18.720000 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.719904    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-service-ca" (OuterVolumeSpecName: "service-ca") pod "fb7986ae-978b-4a9c-a302-d4e18479d9b3" (UID: "fb7986ae-978b-4a9c-a302-d4e18479d9b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:03:18.720174 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.720142    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fb7986ae-978b-4a9c-a302-d4e18479d9b3" (UID: "fb7986ae-978b-4a9c-a302-d4e18479d9b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:03:18.720174 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.720165    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fb7986ae-978b-4a9c-a302-d4e18479d9b3" (UID: "fb7986ae-978b-4a9c-a302-d4e18479d9b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:03:18.721832 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.721805    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fb7986ae-978b-4a9c-a302-d4e18479d9b3" (UID: "fb7986ae-978b-4a9c-a302-d4e18479d9b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:03:18.721935 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.721882    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb7986ae-978b-4a9c-a302-d4e18479d9b3-kube-api-access-r8twc" (OuterVolumeSpecName: "kube-api-access-r8twc") pod "fb7986ae-978b-4a9c-a302-d4e18479d9b3" (UID: "fb7986ae-978b-4a9c-a302-d4e18479d9b3"). InnerVolumeSpecName "kube-api-access-r8twc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:03:18.721935 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.721886    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fb7986ae-978b-4a9c-a302-d4e18479d9b3" (UID: "fb7986ae-978b-4a9c-a302-d4e18479d9b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:03:18.821035 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.820979    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-oauth-config\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:03:18.821035 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.821006    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:03:18.821035 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.821037    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-trusted-ca-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:03:18.821219 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.821046    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8twc\" (UniqueName: \"kubernetes.io/projected/fb7986ae-978b-4a9c-a302-d4e18479d9b3-kube-api-access-r8twc\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:03:18.821219 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.821056    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-oauth-serving-cert\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:03:18.821219 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.821063    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-serving-cert\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:03:18.821219 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:18.821071    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb7986ae-978b-4a9c-a302-d4e18479d9b3-console-config\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:03:19.412338 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:19.412303    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb7986ae-978b-4a9c-a302-d4e18479d9b3" containerID="d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf" exitCode=0
Jul 28 14:03:19.412746 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:19.412349    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595b6c7cb-s4fwb"
Jul 28 14:03:19.412746 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:19.412389    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595b6c7cb-s4fwb" event={"ID":"fb7986ae-978b-4a9c-a302-d4e18479d9b3","Type":"ContainerDied","Data":"d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf"}
Jul 28 14:03:19.412746 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:19.412442    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595b6c7cb-s4fwb" event={"ID":"fb7986ae-978b-4a9c-a302-d4e18479d9b3","Type":"ContainerDied","Data":"ba15bc4b24779ce8ee991a919180d4614b51ceba2b82c661e7a8f5bd4bd7747d"}
Jul 28 14:03:19.412746 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:19.412463    2579 scope.go:117] "RemoveContainer" containerID="d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf"
Jul 28 14:03:19.421553 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:19.421538    2579 scope.go:117] "RemoveContainer" containerID="d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf"
Jul 28 14:03:19.421796 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:03:19.421779    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf\": container with ID starting with d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf not found: ID does not exist" containerID="d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf"
Jul 28 14:03:19.421837 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:19.421803    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf"} err="failed to get container status \"d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf\": rpc error: code = NotFound desc = could not find container \"d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf\": container with ID starting with d554b7334084abf404e5d0b959d5f382e547891c4d270376b6576b60abe827bf not found: ID does not exist"
Jul 28 14:03:19.440428 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:19.440401    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-595b6c7cb-s4fwb"]
Jul 28 14:03:19.445620 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:19.445595    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-595b6c7cb-s4fwb"]
Jul 28 14:03:20.340607 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:20.340572    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb7986ae-978b-4a9c-a302-d4e18479d9b3" path="/var/lib/kubelet/pods/fb7986ae-978b-4a9c-a302-d4e18479d9b3/volumes"
Jul 28 14:03:50.266335 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.266304    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fdb4fcdfd-cmwjq"]
Jul 28 14:03:50.266764 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.266643    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb7986ae-978b-4a9c-a302-d4e18479d9b3" containerName="console"
Jul 28 14:03:50.266764 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.266653    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb7986ae-978b-4a9c-a302-d4e18479d9b3" containerName="console"
Jul 28 14:03:50.266764 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.266718    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb7986ae-978b-4a9c-a302-d4e18479d9b3" containerName="console"
Jul 28 14:03:50.271195 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.271169    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.289621 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.289598    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fdb4fcdfd-cmwjq"]
Jul 28 14:03:50.377601 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.377570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-trusted-ca-bundle\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.377757 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.377609    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-serving-cert\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.377757 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.377625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-oauth-config\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.377757 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.377722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67wh4\" (UniqueName: \"kubernetes.io/projected/9935df18-a3cc-4617-bb38-66678577db20-kube-api-access-67wh4\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.377757 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.377754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-oauth-serving-cert\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.377898 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.377793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-service-ca\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.377898 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.377816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-console-config\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.478915 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.478880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67wh4\" (UniqueName: \"kubernetes.io/projected/9935df18-a3cc-4617-bb38-66678577db20-kube-api-access-67wh4\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.478915 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.478920    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-oauth-serving-cert\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.479200 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.478945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-service-ca\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.479200 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.478967    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-console-config\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.479200 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.479006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-trusted-ca-bundle\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.479344 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.479268    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-serving-cert\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.479344 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.479311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-oauth-config\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.479790 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.479757    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-oauth-serving-cert\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.479916 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.479883    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-console-config\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.479916 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.479892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-trusted-ca-bundle\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.480007 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.479928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-service-ca\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.481978 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.481956    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-serving-cert\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.482042 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.482006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-oauth-config\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.490311 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.490281    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67wh4\" (UniqueName: \"kubernetes.io/projected/9935df18-a3cc-4617-bb38-66678577db20-kube-api-access-67wh4\") pod \"console-5fdb4fcdfd-cmwjq\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") " pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.580784 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.580717    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:03:50.722799 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:50.722778    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fdb4fcdfd-cmwjq"]
Jul 28 14:03:50.724641 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:03:50.724616    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9935df18_a3cc_4617_bb38_66678577db20.slice/crio-35e8a6fd17ff327c6968e0aa7eff541b5b12ddf5306c81cb12cbcf7ea8fdf8af WatchSource:0}: Error finding container 35e8a6fd17ff327c6968e0aa7eff541b5b12ddf5306c81cb12cbcf7ea8fdf8af: Status 404 returned error can't find the container with id 35e8a6fd17ff327c6968e0aa7eff541b5b12ddf5306c81cb12cbcf7ea8fdf8af
Jul 28 14:03:51.547371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:51.547338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb4fcdfd-cmwjq" event={"ID":"9935df18-a3cc-4617-bb38-66678577db20","Type":"ContainerStarted","Data":"9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80"}
Jul 28 14:03:51.547737 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:51.547392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb4fcdfd-cmwjq" event={"ID":"9935df18-a3cc-4617-bb38-66678577db20","Type":"ContainerStarted","Data":"35e8a6fd17ff327c6968e0aa7eff541b5b12ddf5306c81cb12cbcf7ea8fdf8af"}
Jul 28 14:03:51.568330 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:03:51.568283    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fdb4fcdfd-cmwjq" podStartSLOduration=1.568268467 podStartE2EDuration="1.568268467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:03:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:03:51.567089465 +0000 UTC m=+247.694480155" watchObservedRunningTime="2026-07-28 14:03:51.568268467 +0000 UTC m=+247.695659157"
Jul 28 14:04:00.580969 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:00.580936    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:04:00.581458 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:00.580992    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:04:00.585884 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:00.585857    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:04:01.590407 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:01.590381    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:04:01.652203 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:01.652169    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8669f8fc6b-jxq45"]
Jul 28 14:04:26.679434 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.679374    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8669f8fc6b-jxq45" podUID="415f75a8-97f5-4bf3-8170-b333490160f4" containerName="console" containerID="cri-o://8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a" gracePeriod=15
Jul 28 14:04:26.928053 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.928008    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:04:26.988678 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.988594    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7b7zf\" (UniqueName: \"kubernetes.io/projected/415f75a8-97f5-4bf3-8170-b333490160f4-kube-api-access-7b7zf\") pod \"415f75a8-97f5-4bf3-8170-b333490160f4\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") "
Jul 28 14:04:26.988678 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.988634    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-trusted-ca-bundle\") pod \"415f75a8-97f5-4bf3-8170-b333490160f4\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") "
Jul 28 14:04:26.988678 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.988654    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-oauth-config\") pod \"415f75a8-97f5-4bf3-8170-b333490160f4\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") "
Jul 28 14:04:26.988923 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.988683    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-service-ca\") pod \"415f75a8-97f5-4bf3-8170-b333490160f4\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") "
Jul 28 14:04:26.988923 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.988716    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-oauth-serving-cert\") pod \"415f75a8-97f5-4bf3-8170-b333490160f4\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") "
Jul 28 14:04:26.988923 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.988757    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-console-config\") pod \"415f75a8-97f5-4bf3-8170-b333490160f4\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") "
Jul 28 14:04:26.988923 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.988817    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-serving-cert\") pod \"415f75a8-97f5-4bf3-8170-b333490160f4\" (UID: \"415f75a8-97f5-4bf3-8170-b333490160f4\") "
Jul 28 14:04:26.989247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.989200    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "415f75a8-97f5-4bf3-8170-b333490160f4" (UID: "415f75a8-97f5-4bf3-8170-b333490160f4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:04:26.989247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.989209    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-service-ca" (OuterVolumeSpecName: "service-ca") pod "415f75a8-97f5-4bf3-8170-b333490160f4" (UID: "415f75a8-97f5-4bf3-8170-b333490160f4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:04:26.989247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.989217    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "415f75a8-97f5-4bf3-8170-b333490160f4" (UID: "415f75a8-97f5-4bf3-8170-b333490160f4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:04:26.989447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.989273    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-console-config" (OuterVolumeSpecName: "console-config") pod "415f75a8-97f5-4bf3-8170-b333490160f4" (UID: "415f75a8-97f5-4bf3-8170-b333490160f4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:04:26.990987 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.990962    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "415f75a8-97f5-4bf3-8170-b333490160f4" (UID: "415f75a8-97f5-4bf3-8170-b333490160f4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:04:26.991102 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.991079    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "415f75a8-97f5-4bf3-8170-b333490160f4" (UID: "415f75a8-97f5-4bf3-8170-b333490160f4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:04:26.991139 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:26.991117    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/415f75a8-97f5-4bf3-8170-b333490160f4-kube-api-access-7b7zf" (OuterVolumeSpecName: "kube-api-access-7b7zf") pod "415f75a8-97f5-4bf3-8170-b333490160f4" (UID: "415f75a8-97f5-4bf3-8170-b333490160f4"). InnerVolumeSpecName "kube-api-access-7b7zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:04:27.090334 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.090299    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-console-config\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:04:27.090334 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.090329    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-serving-cert\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:04:27.090334 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.090338    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b7zf\" (UniqueName: \"kubernetes.io/projected/415f75a8-97f5-4bf3-8170-b333490160f4-kube-api-access-7b7zf\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:04:27.090561 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.090347    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-trusted-ca-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:04:27.090561 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.090355    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/415f75a8-97f5-4bf3-8170-b333490160f4-console-oauth-config\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:04:27.090561 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.090363    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:04:27.090561 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.090370    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/415f75a8-97f5-4bf3-8170-b333490160f4-oauth-serving-cert\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:04:27.692463 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.692427    2579 generic.go:356] "Generic (PLEG): container finished" podID="415f75a8-97f5-4bf3-8170-b333490160f4" containerID="8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a" exitCode=0
Jul 28 14:04:27.692855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.692479    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669f8fc6b-jxq45"
Jul 28 14:04:27.692855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.692516    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669f8fc6b-jxq45" event={"ID":"415f75a8-97f5-4bf3-8170-b333490160f4","Type":"ContainerDied","Data":"8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a"}
Jul 28 14:04:27.692855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.692568    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669f8fc6b-jxq45" event={"ID":"415f75a8-97f5-4bf3-8170-b333490160f4","Type":"ContainerDied","Data":"89ea75a566f11bdc483e01946cd22eddbcb06adecdc9d00b54c6fdb927e723a0"}
Jul 28 14:04:27.692855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.692588    2579 scope.go:117] "RemoveContainer" containerID="8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a"
Jul 28 14:04:27.701900 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.701881    2579 scope.go:117] "RemoveContainer" containerID="8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a"
Jul 28 14:04:27.702180 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:04:27.702164    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a\": container with ID starting with 8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a not found: ID does not exist" containerID="8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a"
Jul 28 14:04:27.702250 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.702188    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a"} err="failed to get container status \"8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a\": rpc error: code = NotFound desc = could not find container \"8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a\": container with ID starting with 8b3d8f9c51991c0d54c23f37fd03c1dacf70d4bb72bce04dc0ccb5d181b5354a not found: ID does not exist"
Jul 28 14:04:27.718644 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.718613    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8669f8fc6b-jxq45"]
Jul 28 14:04:27.721039 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:27.720992    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8669f8fc6b-jxq45"]
Jul 28 14:04:28.341366 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:28.341329    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="415f75a8-97f5-4bf3-8170-b333490160f4" path="/var/lib/kubelet/pods/415f75a8-97f5-4bf3-8170-b333490160f4/volumes"
Jul 28 14:04:44.183380 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:44.183355    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:04:44.192204 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:44.192177    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:04:44.201525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:04:44.201504    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 14:05:18.462680 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.462646    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"]
Jul 28 14:05:18.464750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.462968    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="415f75a8-97f5-4bf3-8170-b333490160f4" containerName="console"
Jul 28 14:05:18.464750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.462988    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="415f75a8-97f5-4bf3-8170-b333490160f4" containerName="console"
Jul 28 14:05:18.464750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.463079    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="415f75a8-97f5-4bf3-8170-b333490160f4" containerName="console"
Jul 28 14:05:18.466114 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.466098    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.468762 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.468737    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 14:05:18.468884 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.468764    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-r5qv2\""
Jul 28 14:05:18.468884 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.468793    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 14:05:18.477415 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.477392    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"]
Jul 28 14:05:18.524812 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.524774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.524986 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.524818    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msr2w\" (UniqueName: \"kubernetes.io/projected/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-kube-api-access-msr2w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.524986 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.524909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.625354 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.625312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.625544 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.625374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.625544 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.625406    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msr2w\" (UniqueName: \"kubernetes.io/projected/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-kube-api-access-msr2w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.625766 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.625746    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.625850 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.625805    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.636037 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.636003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msr2w\" (UniqueName: \"kubernetes.io/projected/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-kube-api-access-msr2w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.775559 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.775532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:18.915429 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.915359    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"]
Jul 28 14:05:18.918524 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:05:18.918496    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded91989c_d04b_4eb8_9c27_e14dc524bdeb.slice/crio-11a20f3bb1c1dfb4396747981476b627d5a0625cdffedab2b2051eb6f6fa2c8f WatchSource:0}: Error finding container 11a20f3bb1c1dfb4396747981476b627d5a0625cdffedab2b2051eb6f6fa2c8f: Status 404 returned error can't find the container with id 11a20f3bb1c1dfb4396747981476b627d5a0625cdffedab2b2051eb6f6fa2c8f
Jul 28 14:05:18.920600 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:18.920584    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:05:19.911048 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:19.911000    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht" event={"ID":"ed91989c-d04b-4eb8-9c27-e14dc524bdeb","Type":"ContainerStarted","Data":"11a20f3bb1c1dfb4396747981476b627d5a0625cdffedab2b2051eb6f6fa2c8f"}
Jul 28 14:05:24.936299 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:24.936197    2579 generic.go:356] "Generic (PLEG): container finished" podID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerID="a311d29388ee8821e32c1a03a1e0d65541f8b7ce389110b5e9a404aa0940f103" exitCode=0
Jul 28 14:05:24.936299 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:24.936247    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht" event={"ID":"ed91989c-d04b-4eb8-9c27-e14dc524bdeb","Type":"ContainerDied","Data":"a311d29388ee8821e32c1a03a1e0d65541f8b7ce389110b5e9a404aa0940f103"}
Jul 28 14:05:27.956821 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:27.956781    2579 generic.go:356] "Generic (PLEG): container finished" podID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerID="f78c68d04405c048addd86d9fff2f52a750dc0882f72aef64e6a8738c96594ba" exitCode=0
Jul 28 14:05:27.957230 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:27.956874    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht" event={"ID":"ed91989c-d04b-4eb8-9c27-e14dc524bdeb","Type":"ContainerDied","Data":"f78c68d04405c048addd86d9fff2f52a750dc0882f72aef64e6a8738c96594ba"}
Jul 28 14:05:35.995749 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:35.995718    2579 generic.go:356] "Generic (PLEG): container finished" podID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerID="ff06a6b094d0b00cc8445cd598094988b19af322362e79f2a443af9e9185d9a5" exitCode=0
Jul 28 14:05:35.996277 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:35.995766    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht" event={"ID":"ed91989c-d04b-4eb8-9c27-e14dc524bdeb","Type":"ContainerDied","Data":"ff06a6b094d0b00cc8445cd598094988b19af322362e79f2a443af9e9185d9a5"}
Jul 28 14:05:37.128515 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.128489    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:37.296260 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.296225    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msr2w\" (UniqueName: \"kubernetes.io/projected/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-kube-api-access-msr2w\") pod \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") "
Jul 28 14:05:37.296445 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.296352    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-util\") pod \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") "
Jul 28 14:05:37.296445 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.296384    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-bundle\") pod \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\" (UID: \"ed91989c-d04b-4eb8-9c27-e14dc524bdeb\") "
Jul 28 14:05:37.297082 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.297051    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-bundle" (OuterVolumeSpecName: "bundle") pod "ed91989c-d04b-4eb8-9c27-e14dc524bdeb" (UID: "ed91989c-d04b-4eb8-9c27-e14dc524bdeb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:05:37.298499 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.298474    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-kube-api-access-msr2w" (OuterVolumeSpecName: "kube-api-access-msr2w") pod "ed91989c-d04b-4eb8-9c27-e14dc524bdeb" (UID: "ed91989c-d04b-4eb8-9c27-e14dc524bdeb"). InnerVolumeSpecName "kube-api-access-msr2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:05:37.301112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.301087    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-util" (OuterVolumeSpecName: "util") pod "ed91989c-d04b-4eb8-9c27-e14dc524bdeb" (UID: "ed91989c-d04b-4eb8-9c27-e14dc524bdeb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:05:37.397444 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.397407    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-util\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:05:37.397444 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.397433    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:05:37.397444 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:37.397442    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msr2w\" (UniqueName: \"kubernetes.io/projected/ed91989c-d04b-4eb8-9c27-e14dc524bdeb-kube-api-access-msr2w\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:05:38.005900 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:38.005871    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht"
Jul 28 14:05:38.006071 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:38.005904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540flzht" event={"ID":"ed91989c-d04b-4eb8-9c27-e14dc524bdeb","Type":"ContainerDied","Data":"11a20f3bb1c1dfb4396747981476b627d5a0625cdffedab2b2051eb6f6fa2c8f"}
Jul 28 14:05:38.006071 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:38.005950    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11a20f3bb1c1dfb4396747981476b627d5a0625cdffedab2b2051eb6f6fa2c8f"
Jul 28 14:05:41.696131 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.696098    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"]
Jul 28 14:05:41.696520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.696413    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerName="util"
Jul 28 14:05:41.696520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.696423    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerName="util"
Jul 28 14:05:41.696520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.696443    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerName="pull"
Jul 28 14:05:41.696520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.696448    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerName="pull"
Jul 28 14:05:41.696520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.696455    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerName="extract"
Jul 28 14:05:41.696520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.696460    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerName="extract"
Jul 28 14:05:41.696520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.696515    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed91989c-d04b-4eb8-9c27-e14dc524bdeb" containerName="extract"
Jul 28 14:05:41.700168 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.700151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"
Jul 28 14:05:41.702571 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.702550    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 14:05:41.702990 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.702966    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:05:41.703314 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.703299    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-k4m46\""
Jul 28 14:05:41.714909 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.714890    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"]
Jul 28 14:05:41.835868 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.835833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scwqg\" (UniqueName: \"kubernetes.io/projected/c46ca2ba-4882-4e03-8d69-2c325bebe382-kube-api-access-scwqg\") pod \"cert-manager-operator-controller-manager-55c444bf86-nbxm7\" (UID: \"c46ca2ba-4882-4e03-8d69-2c325bebe382\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"
Jul 28 14:05:41.836066 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.835889    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c46ca2ba-4882-4e03-8d69-2c325bebe382-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nbxm7\" (UID: \"c46ca2ba-4882-4e03-8d69-2c325bebe382\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"
Jul 28 14:05:41.936459 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.936428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scwqg\" (UniqueName: \"kubernetes.io/projected/c46ca2ba-4882-4e03-8d69-2c325bebe382-kube-api-access-scwqg\") pod \"cert-manager-operator-controller-manager-55c444bf86-nbxm7\" (UID: \"c46ca2ba-4882-4e03-8d69-2c325bebe382\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"
Jul 28 14:05:41.936623 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.936474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c46ca2ba-4882-4e03-8d69-2c325bebe382-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nbxm7\" (UID: \"c46ca2ba-4882-4e03-8d69-2c325bebe382\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"
Jul 28 14:05:41.936817 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.936802    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c46ca2ba-4882-4e03-8d69-2c325bebe382-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nbxm7\" (UID: \"c46ca2ba-4882-4e03-8d69-2c325bebe382\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"
Jul 28 14:05:41.949691 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:41.949629    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scwqg\" (UniqueName: \"kubernetes.io/projected/c46ca2ba-4882-4e03-8d69-2c325bebe382-kube-api-access-scwqg\") pod \"cert-manager-operator-controller-manager-55c444bf86-nbxm7\" (UID: \"c46ca2ba-4882-4e03-8d69-2c325bebe382\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"
Jul 28 14:05:42.009484 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:42.009456    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"
Jul 28 14:05:42.158469 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:42.158446    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7"]
Jul 28 14:05:42.159665 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:05:42.159639    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46ca2ba_4882_4e03_8d69_2c325bebe382.slice/crio-b7bbd7f6ef143aaeb45869b2f9321532d33a63a876fe7a44e4046f7bcc47e062 WatchSource:0}: Error finding container b7bbd7f6ef143aaeb45869b2f9321532d33a63a876fe7a44e4046f7bcc47e062: Status 404 returned error can't find the container with id b7bbd7f6ef143aaeb45869b2f9321532d33a63a876fe7a44e4046f7bcc47e062
Jul 28 14:05:43.028765 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:43.028729    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7" event={"ID":"c46ca2ba-4882-4e03-8d69-2c325bebe382","Type":"ContainerStarted","Data":"b7bbd7f6ef143aaeb45869b2f9321532d33a63a876fe7a44e4046f7bcc47e062"}
Jul 28 14:05:44.035223 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:44.035181    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7" event={"ID":"c46ca2ba-4882-4e03-8d69-2c325bebe382","Type":"ContainerStarted","Data":"deaf486bb9c48c98acd068778ccc0ce50a727ed0b323efe27127ceb305349ce5"}
Jul 28 14:05:44.069918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:44.069871    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nbxm7" podStartSLOduration=1.484694099 podStartE2EDuration="3.069855681s" totalImagesPullingTime="1.585161582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:05:44.067692626 +0000 UTC m=+360.195083317" watchObservedRunningTime="2026-07-28 14:05:44.069855681 +0000 UTC m=+360.197246372"
Jul 28 14:05:45.609985 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.609950    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"]
Jul 28 14:05:45.613379 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.613361    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.616152 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.616126    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 14:05:45.616252 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.616157    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 14:05:45.617119 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.617108    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-r5qv2\""
Jul 28 14:05:45.624609 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.624588    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"]
Jul 28 14:05:45.648922 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.648894    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x2n5j"]
Jul 28 14:05:45.652086 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.652072    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:05:45.655173 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.655155    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 14:05:45.655272 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.655216    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 14:05:45.655328 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.655314    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-s8mxt\""
Jul 28 14:05:45.666816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.666794    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x2n5j"]
Jul 28 14:05:45.769065 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.769006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51c3a594-0f52-4594-9b96-c142fe110795-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x2n5j\" (UID: \"51c3a594-0f52-4594-9b96-c142fe110795\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:05:45.769237 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.769078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.769237 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.769119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxf9f\" (UniqueName: \"kubernetes.io/projected/da799236-7915-47ac-9313-8dda37a45962-kube-api-access-lxf9f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.769237 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.769138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nh28\" (UniqueName: \"kubernetes.io/projected/51c3a594-0f52-4594-9b96-c142fe110795-kube-api-access-7nh28\") pod \"cert-manager-webhook-6486ffd997-x2n5j\" (UID: \"51c3a594-0f52-4594-9b96-c142fe110795\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:05:45.769237 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.769210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.869857 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.869774    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.869857 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.869827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxf9f\" (UniqueName: \"kubernetes.io/projected/da799236-7915-47ac-9313-8dda37a45962-kube-api-access-lxf9f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.869857 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.869859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nh28\" (UniqueName: \"kubernetes.io/projected/51c3a594-0f52-4594-9b96-c142fe110795-kube-api-access-7nh28\") pod \"cert-manager-webhook-6486ffd997-x2n5j\" (UID: \"51c3a594-0f52-4594-9b96-c142fe110795\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:05:45.870173 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.869911    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.870173 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.869941    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51c3a594-0f52-4594-9b96-c142fe110795-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x2n5j\" (UID: \"51c3a594-0f52-4594-9b96-c142fe110795\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:05:45.870330 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.870304    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.870410 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.870387    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.881293 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.881266    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51c3a594-0f52-4594-9b96-c142fe110795-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x2n5j\" (UID: \"51c3a594-0f52-4594-9b96-c142fe110795\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:05:45.881511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.881491    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nh28\" (UniqueName: \"kubernetes.io/projected/51c3a594-0f52-4594-9b96-c142fe110795-kube-api-access-7nh28\") pod \"cert-manager-webhook-6486ffd997-x2n5j\" (UID: \"51c3a594-0f52-4594-9b96-c142fe110795\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:05:45.881552 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.881491    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxf9f\" (UniqueName: \"kubernetes.io/projected/da799236-7915-47ac-9313-8dda37a45962-kube-api-access-lxf9f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.923184 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.923157    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:45.977574 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:45.977538    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:05:46.084869 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:46.084844    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"]
Jul 28 14:05:46.088223 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:05:46.088195    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda799236_7915_47ac_9313_8dda37a45962.slice/crio-e451ee50468f19d158e83f6a7c7e071ae4c74f2ca2b8c158f09fec877964bc6d WatchSource:0}: Error finding container e451ee50468f19d158e83f6a7c7e071ae4c74f2ca2b8c158f09fec877964bc6d: Status 404 returned error can't find the container with id e451ee50468f19d158e83f6a7c7e071ae4c74f2ca2b8c158f09fec877964bc6d
Jul 28 14:05:46.130855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:46.130773    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x2n5j"]
Jul 28 14:05:46.132694 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:05:46.132668    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51c3a594_0f52_4594_9b96_c142fe110795.slice/crio-1269151db2c25874e71ca1b91f545b8e34f72b91331b314095e93f33398b5327 WatchSource:0}: Error finding container 1269151db2c25874e71ca1b91f545b8e34f72b91331b314095e93f33398b5327: Status 404 returned error can't find the container with id 1269151db2c25874e71ca1b91f545b8e34f72b91331b314095e93f33398b5327
Jul 28 14:05:47.056890 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:47.056851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j" event={"ID":"51c3a594-0f52-4594-9b96-c142fe110795","Type":"ContainerStarted","Data":"1269151db2c25874e71ca1b91f545b8e34f72b91331b314095e93f33398b5327"}
Jul 28 14:05:47.058687 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:47.058659    2579 generic.go:356] "Generic (PLEG): container finished" podID="da799236-7915-47ac-9313-8dda37a45962" containerID="18d0af837143a5e47110d02705244a28cb99f4ee5d8c7604c3feeb009ba65183" exitCode=0
Jul 28 14:05:47.058835 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:47.058712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd" event={"ID":"da799236-7915-47ac-9313-8dda37a45962","Type":"ContainerDied","Data":"18d0af837143a5e47110d02705244a28cb99f4ee5d8c7604c3feeb009ba65183"}
Jul 28 14:05:47.058835 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:47.058749    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd" event={"ID":"da799236-7915-47ac-9313-8dda37a45962","Type":"ContainerStarted","Data":"e451ee50468f19d158e83f6a7c7e071ae4c74f2ca2b8c158f09fec877964bc6d"}
Jul 28 14:05:49.071817 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:49.071782    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j" event={"ID":"51c3a594-0f52-4594-9b96-c142fe110795","Type":"ContainerStarted","Data":"cfe765c064d7410a9c14d93890837aa8e91be63f9649f1a416fd0a2e421fa87f"}
Jul 28 14:05:49.072253 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:49.071915    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:05:49.098742 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:49.098694    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j" podStartSLOduration=1.53305315 podStartE2EDuration="4.098674313s" totalImagesPullingTime="2.565621163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:05:49.096239029 +0000 UTC m=+365.223629722" watchObservedRunningTime="2026-07-28 14:05:49.098674313 +0000 UTC m=+365.226065008"
Jul 28 14:05:51.082814 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:51.082773    2579 generic.go:356] "Generic (PLEG): container finished" podID="da799236-7915-47ac-9313-8dda37a45962" containerID="47f080c9e626ea9b6162c5486f1ac9cdc68491911aba4c44b56c38b51238b243" exitCode=0
Jul 28 14:05:51.083215 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:51.082849    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd" event={"ID":"da799236-7915-47ac-9313-8dda37a45962","Type":"ContainerDied","Data":"47f080c9e626ea9b6162c5486f1ac9cdc68491911aba4c44b56c38b51238b243"}
Jul 28 14:05:52.094801 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:52.094761    2579 generic.go:356] "Generic (PLEG): container finished" podID="da799236-7915-47ac-9313-8dda37a45962" containerID="29cb127693b09f6989e0daa74be070c22f01919a98dad292d900a05f7535294f" exitCode=0
Jul 28 14:05:52.095188 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:52.094808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd" event={"ID":"da799236-7915-47ac-9313-8dda37a45962","Type":"ContainerDied","Data":"29cb127693b09f6989e0daa74be070c22f01919a98dad292d900a05f7535294f"}
Jul 28 14:05:53.235006 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.234969    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:53.337379 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.337344    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxf9f\" (UniqueName: \"kubernetes.io/projected/da799236-7915-47ac-9313-8dda37a45962-kube-api-access-lxf9f\") pod \"da799236-7915-47ac-9313-8dda37a45962\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") "
Jul 28 14:05:53.337379 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.337382    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-bundle\") pod \"da799236-7915-47ac-9313-8dda37a45962\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") "
Jul 28 14:05:53.337577 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.337444    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-util\") pod \"da799236-7915-47ac-9313-8dda37a45962\" (UID: \"da799236-7915-47ac-9313-8dda37a45962\") "
Jul 28 14:05:53.337831 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.337808    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-bundle" (OuterVolumeSpecName: "bundle") pod "da799236-7915-47ac-9313-8dda37a45962" (UID: "da799236-7915-47ac-9313-8dda37a45962"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:05:53.339624 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.339594    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da799236-7915-47ac-9313-8dda37a45962-kube-api-access-lxf9f" (OuterVolumeSpecName: "kube-api-access-lxf9f") pod "da799236-7915-47ac-9313-8dda37a45962" (UID: "da799236-7915-47ac-9313-8dda37a45962"). InnerVolumeSpecName "kube-api-access-lxf9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:05:53.342120 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.342092    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-util" (OuterVolumeSpecName: "util") pod "da799236-7915-47ac-9313-8dda37a45962" (UID: "da799236-7915-47ac-9313-8dda37a45962"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:05:53.438922 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.438835    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-util\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:05:53.438922 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.438864    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxf9f\" (UniqueName: \"kubernetes.io/projected/da799236-7915-47ac-9313-8dda37a45962-kube-api-access-lxf9f\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:05:53.438922 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:53.438878    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da799236-7915-47ac-9313-8dda37a45962-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:05:54.105733 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:54.105698    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd" event={"ID":"da799236-7915-47ac-9313-8dda37a45962","Type":"ContainerDied","Data":"e451ee50468f19d158e83f6a7c7e071ae4c74f2ca2b8c158f09fec877964bc6d"}
Jul 28 14:05:54.105876 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:54.105748    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e451ee50468f19d158e83f6a7c7e071ae4c74f2ca2b8c158f09fec877964bc6d"
Jul 28 14:05:54.105876 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:54.105707    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhncd"
Jul 28 14:05:55.079131 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:05:55.079092    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x2n5j"
Jul 28 14:06:04.886664 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.886629    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"]
Jul 28 14:06:04.887153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.886966    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da799236-7915-47ac-9313-8dda37a45962" containerName="util"
Jul 28 14:06:04.887153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.886984    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="da799236-7915-47ac-9313-8dda37a45962" containerName="util"
Jul 28 14:06:04.887153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.887005    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da799236-7915-47ac-9313-8dda37a45962" containerName="extract"
Jul 28 14:06:04.887153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.887011    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="da799236-7915-47ac-9313-8dda37a45962" containerName="extract"
Jul 28 14:06:04.887153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.887037    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da799236-7915-47ac-9313-8dda37a45962" containerName="pull"
Jul 28 14:06:04.887153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.887042    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="da799236-7915-47ac-9313-8dda37a45962" containerName="pull"
Jul 28 14:06:04.887153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.887095    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="da799236-7915-47ac-9313-8dda37a45962" containerName="extract"
Jul 28 14:06:04.896858 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.896838    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:04.899526 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.899500    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 14:06:04.899658 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.899562    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-r5qv2\""
Jul 28 14:06:04.900339 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.900314    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 14:06:04.900617 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.900591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"]
Jul 28 14:06:04.943609 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.943577    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj9vc\" (UniqueName: \"kubernetes.io/projected/7528aa8d-976c-4148-8dda-067027946841-kube-api-access-vj9vc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:04.943788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.943705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:04.943788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:04.943756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:05.044617 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:05.044578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:05.044808 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:05.044640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj9vc\" (UniqueName: \"kubernetes.io/projected/7528aa8d-976c-4148-8dda-067027946841-kube-api-access-vj9vc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:05.044808 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:05.044775    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:05.044975 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:05.044952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:05.045114 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:05.045095    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:05.059063 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:05.059010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj9vc\" (UniqueName: \"kubernetes.io/projected/7528aa8d-976c-4148-8dda-067027946841-kube-api-access-vj9vc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:05.207712 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:05.207628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:05.358258 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:05.358238    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"]
Jul 28 14:06:05.361169 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:06:05.361137    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7528aa8d_976c_4148_8dda_067027946841.slice/crio-01b023e6fad47613dc49a2bb6c1ae2406f37220a775129aec3376e8235deb4c1 WatchSource:0}: Error finding container 01b023e6fad47613dc49a2bb6c1ae2406f37220a775129aec3376e8235deb4c1: Status 404 returned error can't find the container with id 01b023e6fad47613dc49a2bb6c1ae2406f37220a775129aec3376e8235deb4c1
Jul 28 14:06:06.163057 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:06.163010    2579 generic.go:356] "Generic (PLEG): container finished" podID="7528aa8d-976c-4148-8dda-067027946841" containerID="b199e17df6ce8d52239aee90a7c9610dbb17b194645978d68e9566601b6ac562" exitCode=0
Jul 28 14:06:06.163424 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:06.163052    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k" event={"ID":"7528aa8d-976c-4148-8dda-067027946841","Type":"ContainerDied","Data":"b199e17df6ce8d52239aee90a7c9610dbb17b194645978d68e9566601b6ac562"}
Jul 28 14:06:06.163424 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:06.163104    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k" event={"ID":"7528aa8d-976c-4148-8dda-067027946841","Type":"ContainerStarted","Data":"01b023e6fad47613dc49a2bb6c1ae2406f37220a775129aec3376e8235deb4c1"}
Jul 28 14:06:07.170076 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:07.169962    2579 generic.go:356] "Generic (PLEG): container finished" podID="7528aa8d-976c-4148-8dda-067027946841" containerID="54d38bb0041265ee6d08d7ecf18a831711e3c7e9d00f021144c19ed43ef72dbe" exitCode=0
Jul 28 14:06:07.170076 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:07.170040    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k" event={"ID":"7528aa8d-976c-4148-8dda-067027946841","Type":"ContainerDied","Data":"54d38bb0041265ee6d08d7ecf18a831711e3c7e9d00f021144c19ed43ef72dbe"}
Jul 28 14:06:08.177974 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:08.177937    2579 generic.go:356] "Generic (PLEG): container finished" podID="7528aa8d-976c-4148-8dda-067027946841" containerID="10d57fb9d85d59e05b81ec84ff47a50b96f40c4c7c1a322a1800a247fe10a8ff" exitCode=0
Jul 28 14:06:08.178402 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:08.177988    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k" event={"ID":"7528aa8d-976c-4148-8dda-067027946841","Type":"ContainerDied","Data":"10d57fb9d85d59e05b81ec84ff47a50b96f40c4c7c1a322a1800a247fe10a8ff"}
Jul 28 14:06:09.320594 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.320571    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:09.389543 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.389510    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-util\") pod \"7528aa8d-976c-4148-8dda-067027946841\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") "
Jul 28 14:06:09.389543 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.389568    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-bundle\") pod \"7528aa8d-976c-4148-8dda-067027946841\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") "
Jul 28 14:06:09.389812 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.389621    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj9vc\" (UniqueName: \"kubernetes.io/projected/7528aa8d-976c-4148-8dda-067027946841-kube-api-access-vj9vc\") pod \"7528aa8d-976c-4148-8dda-067027946841\" (UID: \"7528aa8d-976c-4148-8dda-067027946841\") "
Jul 28 14:06:09.390347 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.390323    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-bundle" (OuterVolumeSpecName: "bundle") pod "7528aa8d-976c-4148-8dda-067027946841" (UID: "7528aa8d-976c-4148-8dda-067027946841"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:06:09.391893 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.391872    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7528aa8d-976c-4148-8dda-067027946841-kube-api-access-vj9vc" (OuterVolumeSpecName: "kube-api-access-vj9vc") pod "7528aa8d-976c-4148-8dda-067027946841" (UID: "7528aa8d-976c-4148-8dda-067027946841"). InnerVolumeSpecName "kube-api-access-vj9vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:06:09.397717 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.397693    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-util" (OuterVolumeSpecName: "util") pod "7528aa8d-976c-4148-8dda-067027946841" (UID: "7528aa8d-976c-4148-8dda-067027946841"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:06:09.490298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.490218    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-util\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:09.490298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.490244    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7528aa8d-976c-4148-8dda-067027946841-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:09.490298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:09.490253    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj9vc\" (UniqueName: \"kubernetes.io/projected/7528aa8d-976c-4148-8dda-067027946841-kube-api-access-vj9vc\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:10.190113 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:10.190070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k" event={"ID":"7528aa8d-976c-4148-8dda-067027946841","Type":"ContainerDied","Data":"01b023e6fad47613dc49a2bb6c1ae2406f37220a775129aec3376e8235deb4c1"}
Jul 28 14:06:10.190113 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:10.190097    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjp78k"
Jul 28 14:06:10.190311 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:10.190126    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01b023e6fad47613dc49a2bb6c1ae2406f37220a775129aec3376e8235deb4c1"
Jul 28 14:06:21.001208 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.001172    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"]
Jul 28 14:06:21.001598 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.001498    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7528aa8d-976c-4148-8dda-067027946841" containerName="util"
Jul 28 14:06:21.001598 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.001508    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7528aa8d-976c-4148-8dda-067027946841" containerName="util"
Jul 28 14:06:21.001598 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.001516    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7528aa8d-976c-4148-8dda-067027946841" containerName="pull"
Jul 28 14:06:21.001598 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.001521    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7528aa8d-976c-4148-8dda-067027946841" containerName="pull"
Jul 28 14:06:21.001598 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.001533    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7528aa8d-976c-4148-8dda-067027946841" containerName="extract"
Jul 28 14:06:21.001598 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.001538    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7528aa8d-976c-4148-8dda-067027946841" containerName="extract"
Jul 28 14:06:21.001781 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.001606    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7528aa8d-976c-4148-8dda-067027946841" containerName="extract"
Jul 28 14:06:21.004952 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.004936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.007772 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.007753    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 14:06:21.007901 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.007757    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 14:06:21.007901 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.007756    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fp77x\""
Jul 28 14:06:21.008423 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.008401    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 14:06:21.008511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.008476    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 14:06:21.024591 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.024564    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"]
Jul 28 14:06:21.086501 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.086467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/062f3464-4431-4919-ac0c-d8b2a8b5b562-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.086665 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.086516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/062f3464-4431-4919-ac0c-d8b2a8b5b562-tmp\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.086665 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.086588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/062f3464-4431-4919-ac0c-d8b2a8b5b562-manifests\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.086665 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.086637    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrzcw\" (UniqueName: \"kubernetes.io/projected/062f3464-4431-4919-ac0c-d8b2a8b5b562-kube-api-access-jrzcw\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.086875 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.086706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/062f3464-4431-4919-ac0c-d8b2a8b5b562-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.187175 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.187144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/062f3464-4431-4919-ac0c-d8b2a8b5b562-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.187175 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.187191    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/062f3464-4431-4919-ac0c-d8b2a8b5b562-tmp\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.187500 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.187217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/062f3464-4431-4919-ac0c-d8b2a8b5b562-manifests\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.187500 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.187236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrzcw\" (UniqueName: \"kubernetes.io/projected/062f3464-4431-4919-ac0c-d8b2a8b5b562-kube-api-access-jrzcw\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.187500 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.187264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/062f3464-4431-4919-ac0c-d8b2a8b5b562-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.187767 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.187703    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/062f3464-4431-4919-ac0c-d8b2a8b5b562-manifests\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.189771 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.189746    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/062f3464-4431-4919-ac0c-d8b2a8b5b562-tmp\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.189976 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.189958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/062f3464-4431-4919-ac0c-d8b2a8b5b562-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.190122 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.190103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/062f3464-4431-4919-ac0c-d8b2a8b5b562-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.198844 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.198816    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrzcw\" (UniqueName: \"kubernetes.io/projected/062f3464-4431-4919-ac0c-d8b2a8b5b562-kube-api-access-jrzcw\") pod \"opendatahub-operator-controller-manager-6b59958787-mpjlj\" (UID: \"062f3464-4431-4919-ac0c-d8b2a8b5b562\") " pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.315551 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.315510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:21.484965 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:21.484925    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"]
Jul 28 14:06:21.488676 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:06:21.488648    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod062f3464_4431_4919_ac0c_d8b2a8b5b562.slice/crio-4f58ae71747cab07b38ddb2caec13ec7028b9820b380585abbe0f4db21004739 WatchSource:0}: Error finding container 4f58ae71747cab07b38ddb2caec13ec7028b9820b380585abbe0f4db21004739: Status 404 returned error can't find the container with id 4f58ae71747cab07b38ddb2caec13ec7028b9820b380585abbe0f4db21004739
Jul 28 14:06:22.252110 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:22.252060    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj" event={"ID":"062f3464-4431-4919-ac0c-d8b2a8b5b562","Type":"ContainerStarted","Data":"4f58ae71747cab07b38ddb2caec13ec7028b9820b380585abbe0f4db21004739"}
Jul 28 14:06:25.270218 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:25.270181    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj" event={"ID":"062f3464-4431-4919-ac0c-d8b2a8b5b562","Type":"ContainerStarted","Data":"85faa73c63db2ac648b0658b6dc66d3530ae3027d01dc9c221508498f29addb6"}
Jul 28 14:06:26.276735 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:26.276700    2579 generic.go:356] "Generic (PLEG): container finished" podID="062f3464-4431-4919-ac0c-d8b2a8b5b562" containerID="85faa73c63db2ac648b0658b6dc66d3530ae3027d01dc9c221508498f29addb6" exitCode=0
Jul 28 14:06:26.277141 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:26.276769    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj" event={"ID":"062f3464-4431-4919-ac0c-d8b2a8b5b562","Type":"ContainerDied","Data":"85faa73c63db2ac648b0658b6dc66d3530ae3027d01dc9c221508498f29addb6"}
Jul 28 14:06:27.283094 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:27.283055    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj" event={"ID":"062f3464-4431-4919-ac0c-d8b2a8b5b562","Type":"ContainerStarted","Data":"6f1ec32be403059146098b81af77e349bc37e93ebeafc74cd4b59dc685c7e7e2"}
Jul 28 14:06:27.283512 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:27.283124    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:27.312349 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:27.312285    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj" podStartSLOduration=2.60321827 podStartE2EDuration="7.312266171s" totalImagesPullingTime="3.644120685s" totalInitContainerRuntime="1.064927216s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:06:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:06:27.311006328 +0000 UTC m=+403.438397019" watchObservedRunningTime="2026-07-28 14:06:27.312266171 +0000 UTC m=+403.439656863"
Jul 28 14:06:38.291609 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:38.291577    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b59958787-mpjlj"
Jul 28 14:06:41.477218 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.477179    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-97487846c-7xmvp"]
Jul 28 14:06:41.484618 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.484595    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.488330 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.488292    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 14:06:41.488452 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.488303    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xhjnq\""
Jul 28 14:06:41.488452 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.488401    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 14:06:41.503538 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.503505    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-97487846c-7xmvp"]
Jul 28 14:06:41.574632 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.574600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef918895-4378-4d1d-8d5d-4f47f4595333-tmp\") pod \"kube-auth-proxy-97487846c-7xmvp\" (UID: \"ef918895-4378-4d1d-8d5d-4f47f4595333\") " pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.574632 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.574632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ld7k\" (UniqueName: \"kubernetes.io/projected/ef918895-4378-4d1d-8d5d-4f47f4595333-kube-api-access-4ld7k\") pod \"kube-auth-proxy-97487846c-7xmvp\" (UID: \"ef918895-4378-4d1d-8d5d-4f47f4595333\") " pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.574822 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.574661    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef918895-4378-4d1d-8d5d-4f47f4595333-tls-certs\") pod \"kube-auth-proxy-97487846c-7xmvp\" (UID: \"ef918895-4378-4d1d-8d5d-4f47f4595333\") " pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.594662 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.594627    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d8ctb"]
Jul 28 14:06:41.597964 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.597943    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:41.600636 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.600616    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 14:06:41.600843 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.600826    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ntlwl\""
Jul 28 14:06:41.617192 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.617157    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d8ctb"]
Jul 28 14:06:41.675637 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.675606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/870a4744-f59d-40ea-8b33-ce48af9afa40-cert\") pod \"odh-model-controller-7998c67dc4-d8ctb\" (UID: \"870a4744-f59d-40ea-8b33-ce48af9afa40\") " pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:41.675801 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.675706    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef918895-4378-4d1d-8d5d-4f47f4595333-tmp\") pod \"kube-auth-proxy-97487846c-7xmvp\" (UID: \"ef918895-4378-4d1d-8d5d-4f47f4595333\") " pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.675801 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.675731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ld7k\" (UniqueName: \"kubernetes.io/projected/ef918895-4378-4d1d-8d5d-4f47f4595333-kube-api-access-4ld7k\") pod \"kube-auth-proxy-97487846c-7xmvp\" (UID: \"ef918895-4378-4d1d-8d5d-4f47f4595333\") " pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.675801 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.675765    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef918895-4378-4d1d-8d5d-4f47f4595333-tls-certs\") pod \"kube-auth-proxy-97487846c-7xmvp\" (UID: \"ef918895-4378-4d1d-8d5d-4f47f4595333\") " pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.675801 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.675783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgvvc\" (UniqueName: \"kubernetes.io/projected/870a4744-f59d-40ea-8b33-ce48af9afa40-kube-api-access-rgvvc\") pod \"odh-model-controller-7998c67dc4-d8ctb\" (UID: \"870a4744-f59d-40ea-8b33-ce48af9afa40\") " pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:41.678181 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.678154    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef918895-4378-4d1d-8d5d-4f47f4595333-tmp\") pod \"kube-auth-proxy-97487846c-7xmvp\" (UID: \"ef918895-4378-4d1d-8d5d-4f47f4595333\") " pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.678431 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.678411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef918895-4378-4d1d-8d5d-4f47f4595333-tls-certs\") pod \"kube-auth-proxy-97487846c-7xmvp\" (UID: \"ef918895-4378-4d1d-8d5d-4f47f4595333\") " pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.692495 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.692469    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ld7k\" (UniqueName: \"kubernetes.io/projected/ef918895-4378-4d1d-8d5d-4f47f4595333-kube-api-access-4ld7k\") pod \"kube-auth-proxy-97487846c-7xmvp\" (UID: \"ef918895-4378-4d1d-8d5d-4f47f4595333\") " pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.776931 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.776906    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvvc\" (UniqueName: \"kubernetes.io/projected/870a4744-f59d-40ea-8b33-ce48af9afa40-kube-api-access-rgvvc\") pod \"odh-model-controller-7998c67dc4-d8ctb\" (UID: \"870a4744-f59d-40ea-8b33-ce48af9afa40\") " pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:41.777106 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.776942    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/870a4744-f59d-40ea-8b33-ce48af9afa40-cert\") pod \"odh-model-controller-7998c67dc4-d8ctb\" (UID: \"870a4744-f59d-40ea-8b33-ce48af9afa40\") " pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:41.777106 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:06:41.777076    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 14:06:41.777179 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:06:41.777132    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/870a4744-f59d-40ea-8b33-ce48af9afa40-cert podName:870a4744-f59d-40ea-8b33-ce48af9afa40 nodeName:}" failed. No retries permitted until 2026-07-28 14:06:42.277116191 +0000 UTC m=+418.404506860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/870a4744-f59d-40ea-8b33-ce48af9afa40-cert") pod "odh-model-controller-7998c67dc4-d8ctb" (UID: "870a4744-f59d-40ea-8b33-ce48af9afa40") : secret "odh-model-controller-webhook-cert" not found
Jul 28 14:06:41.791352 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.791325    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgvvc\" (UniqueName: \"kubernetes.io/projected/870a4744-f59d-40ea-8b33-ce48af9afa40-kube-api-access-rgvvc\") pod \"odh-model-controller-7998c67dc4-d8ctb\" (UID: \"870a4744-f59d-40ea-8b33-ce48af9afa40\") " pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:41.794106 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.794092    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp"
Jul 28 14:06:41.951094 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:41.951063    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-97487846c-7xmvp"]
Jul 28 14:06:41.953981 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:06:41.953957    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef918895_4378_4d1d_8d5d_4f47f4595333.slice/crio-6a82b0e85599f5d9c3f3321e798d0c943e98be955a5762226a5d5be797b8a90f WatchSource:0}: Error finding container 6a82b0e85599f5d9c3f3321e798d0c943e98be955a5762226a5d5be797b8a90f: Status 404 returned error can't find the container with id 6a82b0e85599f5d9c3f3321e798d0c943e98be955a5762226a5d5be797b8a90f
Jul 28 14:06:42.282832 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.282799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/870a4744-f59d-40ea-8b33-ce48af9afa40-cert\") pod \"odh-model-controller-7998c67dc4-d8ctb\" (UID: \"870a4744-f59d-40ea-8b33-ce48af9afa40\") " pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:42.285314 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.285295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/870a4744-f59d-40ea-8b33-ce48af9afa40-cert\") pod \"odh-model-controller-7998c67dc4-d8ctb\" (UID: \"870a4744-f59d-40ea-8b33-ce48af9afa40\") " pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:42.368336 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.368296    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp" event={"ID":"ef918895-4378-4d1d-8d5d-4f47f4595333","Type":"ContainerStarted","Data":"6a82b0e85599f5d9c3f3321e798d0c943e98be955a5762226a5d5be797b8a90f"}
Jul 28 14:06:42.477069 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.477035    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lj2hf"]
Jul 28 14:06:42.480902 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.480878    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:06:42.483984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.483951    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 14:06:42.483984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.483962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-h4bg5\""
Jul 28 14:06:42.496978 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.496956    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lj2hf"]
Jul 28 14:06:42.508362 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.508338    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:42.585128 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.585098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b2609ab-ae57-4514-a372-1c05a74b180b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lj2hf\" (UID: \"0b2609ab-ae57-4514-a372-1c05a74b180b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:06:42.585296 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.585135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpcx2\" (UniqueName: \"kubernetes.io/projected/0b2609ab-ae57-4514-a372-1c05a74b180b-kube-api-access-kpcx2\") pod \"model-serving-api-54d89bd5fc-lj2hf\" (UID: \"0b2609ab-ae57-4514-a372-1c05a74b180b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:06:42.683343 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.683300    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d8ctb"]
Jul 28 14:06:42.685901 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.685869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b2609ab-ae57-4514-a372-1c05a74b180b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lj2hf\" (UID: \"0b2609ab-ae57-4514-a372-1c05a74b180b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:06:42.686054 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.685922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpcx2\" (UniqueName: \"kubernetes.io/projected/0b2609ab-ae57-4514-a372-1c05a74b180b-kube-api-access-kpcx2\") pod \"model-serving-api-54d89bd5fc-lj2hf\" (UID: \"0b2609ab-ae57-4514-a372-1c05a74b180b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:06:42.686257 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:06:42.686233    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod870a4744_f59d_40ea_8b33_ce48af9afa40.slice/crio-6e11b5e5f80476fc527d1a10ed4f2a3e3f69a4e228462e43b79006d6c782c0f6 WatchSource:0}: Error finding container 6e11b5e5f80476fc527d1a10ed4f2a3e3f69a4e228462e43b79006d6c782c0f6: Status 404 returned error can't find the container with id 6e11b5e5f80476fc527d1a10ed4f2a3e3f69a4e228462e43b79006d6c782c0f6
Jul 28 14:06:42.689060 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.689038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b2609ab-ae57-4514-a372-1c05a74b180b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lj2hf\" (UID: \"0b2609ab-ae57-4514-a372-1c05a74b180b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:06:42.706311 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.706283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpcx2\" (UniqueName: \"kubernetes.io/projected/0b2609ab-ae57-4514-a372-1c05a74b180b-kube-api-access-kpcx2\") pod \"model-serving-api-54d89bd5fc-lj2hf\" (UID: \"0b2609ab-ae57-4514-a372-1c05a74b180b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:06:42.795250 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.795214    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:06:42.955079 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:42.955046    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lj2hf"]
Jul 28 14:06:42.959369 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:06:42.959338    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b2609ab_ae57_4514_a372_1c05a74b180b.slice/crio-8064e5ed27516ee10a874f455b35b76a592f80a2eb9743e2d61f0a2df228dd4d WatchSource:0}: Error finding container 8064e5ed27516ee10a874f455b35b76a592f80a2eb9743e2d61f0a2df228dd4d: Status 404 returned error can't find the container with id 8064e5ed27516ee10a874f455b35b76a592f80a2eb9743e2d61f0a2df228dd4d
Jul 28 14:06:43.248720 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.248598    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"]
Jul 28 14:06:43.254257 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.254231    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.258686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.258660    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 14:06:43.258851 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.258758    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 14:06:43.258851 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.258662    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c9lb6\""
Jul 28 14:06:43.258851 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.258842    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 14:06:43.259122 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.259105    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 14:06:43.276672 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.276643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"]
Jul 28 14:06:43.291130 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.291103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9jrz\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-kube-api-access-t9jrz\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.291269 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.291152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.291269 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.291227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bffc89b0-d93a-4847-9b46-5afab8afcf35-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.291269 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.291262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.291415 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.291280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.291415 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.291307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.291511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.291458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.377882 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.377849    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb" event={"ID":"870a4744-f59d-40ea-8b33-ce48af9afa40","Type":"ContainerStarted","Data":"6e11b5e5f80476fc527d1a10ed4f2a3e3f69a4e228462e43b79006d6c782c0f6"}
Jul 28 14:06:43.379702 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.379672    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf" event={"ID":"0b2609ab-ae57-4514-a372-1c05a74b180b","Type":"ContainerStarted","Data":"8064e5ed27516ee10a874f455b35b76a592f80a2eb9743e2d61f0a2df228dd4d"}
Jul 28 14:06:43.392294 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.392265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.392516 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.392332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bffc89b0-d93a-4847-9b46-5afab8afcf35-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.392516 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.392361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.392516 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.392385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.392516 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.392424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.392516 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.392493    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.392749 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.392545    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9jrz\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-kube-api-access-t9jrz\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.393388 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.393363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.395182 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.395120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.395327 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.395309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bffc89b0-d93a-4847-9b46-5afab8afcf35-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.395532 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.395512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.396284 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.396256    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.402774 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.402751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9jrz\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-kube-api-access-t9jrz\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.403520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.403502    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-h8kff\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.569969 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.569936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:43.773203 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:43.773133    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"]
Jul 28 14:06:45.349609 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:06:45.349574    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbffc89b0_d93a_4847_9b46_5afab8afcf35.slice/crio-f0b0fe340abd3d4eef898520485f0c550935dd7a8ebca8514c13218479a8dfca WatchSource:0}: Error finding container f0b0fe340abd3d4eef898520485f0c550935dd7a8ebca8514c13218479a8dfca: Status 404 returned error can't find the container with id f0b0fe340abd3d4eef898520485f0c550935dd7a8ebca8514c13218479a8dfca
Jul 28 14:06:45.397520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:45.397485    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff" event={"ID":"bffc89b0-d93a-4847-9b46-5afab8afcf35","Type":"ContainerStarted","Data":"f0b0fe340abd3d4eef898520485f0c550935dd7a8ebca8514c13218479a8dfca"}
Jul 28 14:06:46.325116 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:46.325094    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 14:06:47.412253 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:47.412222    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb" event={"ID":"870a4744-f59d-40ea-8b33-ce48af9afa40","Type":"ContainerStarted","Data":"cabfc0e0e3fea8ee0730b1c0d9902ded1fa87c015d4a8775dcf9d973e0252d7a"}
Jul 28 14:06:47.412689 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:47.412397    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:47.414150 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:47.414119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf" event={"ID":"0b2609ab-ae57-4514-a372-1c05a74b180b","Type":"ContainerStarted","Data":"f88acbeb875c03156c05a899e041c5fc8ae2d568dfe4dcd88a0ba96599af0f54"}
Jul 28 14:06:47.414283 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:47.414180    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:06:47.416065 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:47.416011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp" event={"ID":"ef918895-4378-4d1d-8d5d-4f47f4595333","Type":"ContainerStarted","Data":"4e82b8b3fc9a98727161e5d103a732a1100ec33dec8fa9a8194b33d0b8d30292"}
Jul 28 14:06:47.441737 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:47.441694    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb" podStartSLOduration=2.79687024 podStartE2EDuration="6.441679653s" totalImagesPullingTime="3.644809413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:06:47.441282084 +0000 UTC m=+423.568672779" watchObservedRunningTime="2026-07-28 14:06:47.441679653 +0000 UTC m=+423.569070344"
Jul 28 14:06:47.527619 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:47.527501    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf" podStartSLOduration=2.16674201 podStartE2EDuration="5.52747722s" totalImagesPullingTime="3.36073521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:06:47.484297583 +0000 UTC m=+423.611688278" watchObservedRunningTime="2026-07-28 14:06:47.52747722 +0000 UTC m=+423.654867912"
Jul 28 14:06:47.528426 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:47.528388    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-97487846c-7xmvp" podStartSLOduration=2.161649232 podStartE2EDuration="6.528378759s" totalImagesPullingTime="4.366729527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:06:47.52458049 +0000 UTC m=+423.651971191" watchObservedRunningTime="2026-07-28 14:06:47.528378759 +0000 UTC m=+423.655769452"
Jul 28 14:06:48.186053 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:48.185994    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 14:06:48.186136 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:48.186086    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 14:06:48.422804 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:48.422760    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff" event={"ID":"bffc89b0-d93a-4847-9b46-5afab8afcf35","Type":"ContainerStarted","Data":"be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758"}
Jul 28 14:06:48.423298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:48.423257    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:48.477067 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:48.476944    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff" podStartSLOduration=2.642834636 podStartE2EDuration="5.476926823s" totalImagesPullingTime="2.834092187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:06:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:06:48.474470498 +0000 UTC m=+424.601861190" watchObservedRunningTime="2026-07-28 14:06:48.476926823 +0000 UTC m=+424.604317516"
Jul 28 14:06:49.430354 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:49.430322    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-h8kff container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 14:06:49.430717 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:49.430381    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff" podUID="bffc89b0-d93a-4847-9b46-5afab8afcf35" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 14:06:50.436831 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.436802    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:50.710800 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.710728    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84947845b-klwsc"]
Jul 28 14:06:50.714394 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.714379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.717183 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.717145    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 14:06:50.717644 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.717626    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 14:06:50.717767 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.717749    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:06:50.718205 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.718183    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 14:06:50.718432 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.718399    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 14:06:50.718554 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.718536    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-qmc27\""
Jul 28 14:06:50.725071 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.725049    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84947845b-klwsc"]
Jul 28 14:06:50.765528 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.765496    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d76efeb6-b8a6-423f-92e6-99400026d3c7-metrics-cert\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.765681 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.765540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d76efeb6-b8a6-423f-92e6-99400026d3c7-cert\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.765681 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.765601    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5jsr\" (UniqueName: \"kubernetes.io/projected/d76efeb6-b8a6-423f-92e6-99400026d3c7-kube-api-access-w5jsr\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.765681 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.765667    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d76efeb6-b8a6-423f-92e6-99400026d3c7-manager-config\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.866523 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.866489    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d76efeb6-b8a6-423f-92e6-99400026d3c7-metrics-cert\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.866523 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.866526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d76efeb6-b8a6-423f-92e6-99400026d3c7-cert\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.866777 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.866557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5jsr\" (UniqueName: \"kubernetes.io/projected/d76efeb6-b8a6-423f-92e6-99400026d3c7-kube-api-access-w5jsr\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.866777 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.866599    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d76efeb6-b8a6-423f-92e6-99400026d3c7-manager-config\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.867409 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.867318    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d76efeb6-b8a6-423f-92e6-99400026d3c7-manager-config\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.869105 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.869077    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d76efeb6-b8a6-423f-92e6-99400026d3c7-metrics-cert\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.869105 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.869094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d76efeb6-b8a6-423f-92e6-99400026d3c7-cert\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:50.878634 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:50.878610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5jsr\" (UniqueName: \"kubernetes.io/projected/d76efeb6-b8a6-423f-92e6-99400026d3c7-kube-api-access-w5jsr\") pod \"lws-controller-manager-84947845b-klwsc\" (UID: \"d76efeb6-b8a6-423f-92e6-99400026d3c7\") " pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:51.025603 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:51.025573    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:51.182868 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:51.182846    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84947845b-klwsc"]
Jul 28 14:06:51.184760 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:06:51.184737    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76efeb6_b8a6_423f_92e6_99400026d3c7.slice/crio-26ffff705357720ba20dc53b5da6e3a374a261fb470d065e072d7c4065cb3c75 WatchSource:0}: Error finding container 26ffff705357720ba20dc53b5da6e3a374a261fb470d065e072d7c4065cb3c75: Status 404 returned error can't find the container with id 26ffff705357720ba20dc53b5da6e3a374a261fb470d065e072d7c4065cb3c75
Jul 28 14:06:51.442673 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:51.442593    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc" event={"ID":"d76efeb6-b8a6-423f-92e6-99400026d3c7","Type":"ContainerStarted","Data":"26ffff705357720ba20dc53b5da6e3a374a261fb470d065e072d7c4065cb3c75"}
Jul 28 14:06:53.920428 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:53.920389    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"]
Jul 28 14:06:53.920889 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:53.920606    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff" podUID="bffc89b0-d93a-4847-9b46-5afab8afcf35" containerName="discovery" containerID="cri-o://be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758" gracePeriod=30
Jul 28 14:06:54.162395 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.162372    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:54.300755 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.300731    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-cacerts\") pod \"bffc89b0-d93a-4847-9b46-5afab8afcf35\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") "
Jul 28 14:06:54.300936 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.300815    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-token\") pod \"bffc89b0-d93a-4847-9b46-5afab8afcf35\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") "
Jul 28 14:06:54.300936 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.300837    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-kubeconfig\") pod \"bffc89b0-d93a-4847-9b46-5afab8afcf35\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") "
Jul 28 14:06:54.300936 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.300880    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-ca-configmap\") pod \"bffc89b0-d93a-4847-9b46-5afab8afcf35\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") "
Jul 28 14:06:54.301148 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.300933    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-dns-cert\") pod \"bffc89b0-d93a-4847-9b46-5afab8afcf35\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") "
Jul 28 14:06:54.301148 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.300970    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9jrz\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-kube-api-access-t9jrz\") pod \"bffc89b0-d93a-4847-9b46-5afab8afcf35\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") "
Jul 28 14:06:54.301148 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.300993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bffc89b0-d93a-4847-9b46-5afab8afcf35-local-certs\") pod \"bffc89b0-d93a-4847-9b46-5afab8afcf35\" (UID: \"bffc89b0-d93a-4847-9b46-5afab8afcf35\") "
Jul 28 14:06:54.301413 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.301381    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "bffc89b0-d93a-4847-9b46-5afab8afcf35" (UID: "bffc89b0-d93a-4847-9b46-5afab8afcf35"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:06:54.303465 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.303442    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-cacerts" (OuterVolumeSpecName: "cacerts") pod "bffc89b0-d93a-4847-9b46-5afab8afcf35" (UID: "bffc89b0-d93a-4847-9b46-5afab8afcf35"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:06:54.303465 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.303455    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "bffc89b0-d93a-4847-9b46-5afab8afcf35" (UID: "bffc89b0-d93a-4847-9b46-5afab8afcf35"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:06:54.303750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.303717    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-token" (OuterVolumeSpecName: "istio-token") pod "bffc89b0-d93a-4847-9b46-5afab8afcf35" (UID: "bffc89b0-d93a-4847-9b46-5afab8afcf35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:06:54.303842 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.303763    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bffc89b0-d93a-4847-9b46-5afab8afcf35-local-certs" (OuterVolumeSpecName: "local-certs") pod "bffc89b0-d93a-4847-9b46-5afab8afcf35" (UID: "bffc89b0-d93a-4847-9b46-5afab8afcf35"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:06:54.303842 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.303773    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "bffc89b0-d93a-4847-9b46-5afab8afcf35" (UID: "bffc89b0-d93a-4847-9b46-5afab8afcf35"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:06:54.303919 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.303862    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-kube-api-access-t9jrz" (OuterVolumeSpecName: "kube-api-access-t9jrz") pod "bffc89b0-d93a-4847-9b46-5afab8afcf35" (UID: "bffc89b0-d93a-4847-9b46-5afab8afcf35"). InnerVolumeSpecName "kube-api-access-t9jrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:06:54.402633 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.402602    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-token\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:54.402633 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.402625    2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-kubeconfig\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:54.402633 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.402635    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-ca-configmap\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:54.402860 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.402643    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-istio-csr-dns-cert\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:54.402860 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.402653    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9jrz\" (UniqueName: \"kubernetes.io/projected/bffc89b0-d93a-4847-9b46-5afab8afcf35-kube-api-access-t9jrz\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:54.402860 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.402661    2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bffc89b0-d93a-4847-9b46-5afab8afcf35-local-certs\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:54.402860 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.402668    2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bffc89b0-d93a-4847-9b46-5afab8afcf35-cacerts\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:06:54.461620 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.461577    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc" event={"ID":"d76efeb6-b8a6-423f-92e6-99400026d3c7","Type":"ContainerStarted","Data":"19b8763bf83677a258d109a1f6231087220d40bcbe913b96cf5d3f8a442e5d53"}
Jul 28 14:06:54.461809 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.461686    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:06:54.462960 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.462929    2579 generic.go:356] "Generic (PLEG): container finished" podID="bffc89b0-d93a-4847-9b46-5afab8afcf35" containerID="be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758" exitCode=0
Jul 28 14:06:54.463108 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.462978    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"
Jul 28 14:06:54.463108 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.463032    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff" event={"ID":"bffc89b0-d93a-4847-9b46-5afab8afcf35","Type":"ContainerDied","Data":"be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758"}
Jul 28 14:06:54.463108 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.463077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff" event={"ID":"bffc89b0-d93a-4847-9b46-5afab8afcf35","Type":"ContainerDied","Data":"f0b0fe340abd3d4eef898520485f0c550935dd7a8ebca8514c13218479a8dfca"}
Jul 28 14:06:54.463108 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.463097    2579 scope.go:117] "RemoveContainer" containerID="be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758"
Jul 28 14:06:54.471808 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.471790    2579 scope.go:117] "RemoveContainer" containerID="be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758"
Jul 28 14:06:54.472095 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:06:54.472075    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758\": container with ID starting with be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758 not found: ID does not exist" containerID="be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758"
Jul 28 14:06:54.472177 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.472101    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758"} err="failed to get container status \"be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758\": rpc error: code = NotFound desc = could not find container \"be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758\": container with ID starting with be6eef5fa53280366df8cd4d02c9a78edb2644d217caa36a4ac5f4cee9d4d758 not found: ID does not exist"
Jul 28 14:06:54.509293 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.509231    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc" podStartSLOduration=2.290809039 podStartE2EDuration="4.509210322s" totalImagesPullingTime="2.218401283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:06:54.506192626 +0000 UTC m=+430.633583317" watchObservedRunningTime="2026-07-28 14:06:54.509210322 +0000 UTC m=+430.636601017"
Jul 28 14:06:54.529100 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.529067    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"]
Jul 28 14:06:54.549522 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:54.549497    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-h8kff"]
Jul 28 14:06:56.341984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:56.341948    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bffc89b0-d93a-4847-9b46-5afab8afcf35" path="/var/lib/kubelet/pods/bffc89b0-d93a-4847-9b46-5afab8afcf35/volumes"
Jul 28 14:06:58.424973 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:58.424942    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-d8ctb"
Jul 28 14:06:58.426943 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:06:58.426924    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lj2hf"
Jul 28 14:07:05.471645 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:05.471572    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84947845b-klwsc"
Jul 28 14:07:58.164006 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.163969    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"]
Jul 28 14:07:58.164477 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.164353    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bffc89b0-d93a-4847-9b46-5afab8afcf35" containerName="discovery"
Jul 28 14:07:58.164477 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.164364    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bffc89b0-d93a-4847-9b46-5afab8afcf35" containerName="discovery"
Jul 28 14:07:58.164477 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.164445    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bffc89b0-d93a-4847-9b46-5afab8afcf35" containerName="discovery"
Jul 28 14:07:58.168986 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.168968    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.171654 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.171632    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wwwfv\""
Jul 28 14:07:58.171742 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.171690    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 14:07:58.172342 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.172328    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 14:07:58.184086 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.184066    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"]
Jul 28 14:07:58.244535 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.244509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.244650 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.244560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjk5\" (UniqueName: \"kubernetes.io/projected/1ca6eeb4-4269-4ec0-a231-da61c5816719-kube-api-access-lsjk5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.244650 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.244583    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.345979 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.345947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.346147 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.346043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjk5\" (UniqueName: \"kubernetes.io/projected/1ca6eeb4-4269-4ec0-a231-da61c5816719-kube-api-access-lsjk5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.346147 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.346084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.346331 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.346309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.346425 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.346406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.356725 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.356695    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjk5\" (UniqueName: \"kubernetes.io/projected/1ca6eeb4-4269-4ec0-a231-da61c5816719-kube-api-access-lsjk5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.478402 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.478335    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:07:58.631700 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.631652    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"]
Jul 28 14:07:58.634162 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:07:58.634135    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca6eeb4_4269_4ec0_a231_da61c5816719.slice/crio-0513290dbc958509528270a67fd85b79660886ac3739e47e89931871ebaea268 WatchSource:0}: Error finding container 0513290dbc958509528270a67fd85b79660886ac3739e47e89931871ebaea268: Status 404 returned error can't find the container with id 0513290dbc958509528270a67fd85b79660886ac3739e47e89931871ebaea268
Jul 28 14:07:58.783625 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.783589    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"]
Jul 28 14:07:58.788624 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.788599    2579 generic.go:356] "Generic (PLEG): container finished" podID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerID="11221321eedb8594fff9c85c543347917db4494a17a67bd883e422590f6c1fa7" exitCode=0
Jul 28 14:07:58.789552 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.789489    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c" event={"ID":"1ca6eeb4-4269-4ec0-a231-da61c5816719","Type":"ContainerDied","Data":"11221321eedb8594fff9c85c543347917db4494a17a67bd883e422590f6c1fa7"}
Jul 28 14:07:58.789552 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.789519    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c" event={"ID":"1ca6eeb4-4269-4ec0-a231-da61c5816719","Type":"ContainerStarted","Data":"0513290dbc958509528270a67fd85b79660886ac3739e47e89931871ebaea268"}
Jul 28 14:07:58.789663 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.789615    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:58.795395 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.795371    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"]
Jul 28 14:07:58.951122 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.951094    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:58.951317 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.951148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:58.951317 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:58.951212    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv7qh\" (UniqueName: \"kubernetes.io/projected/54c9af46-5b09-4b16-a585-3de8baa9d6b9-kube-api-access-bv7qh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:59.051972 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.051941    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv7qh\" (UniqueName: \"kubernetes.io/projected/54c9af46-5b09-4b16-a585-3de8baa9d6b9-kube-api-access-bv7qh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:59.052169 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.051988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:59.052169 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.052052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:59.052378 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.052360    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:59.052439 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.052397    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:59.062233 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.062214    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv7qh\" (UniqueName: \"kubernetes.io/projected/54c9af46-5b09-4b16-a585-3de8baa9d6b9-kube-api-access-bv7qh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:59.103167 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.103143    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:07:59.241671 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.241645    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"]
Jul 28 14:07:59.243873 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:07:59.243846    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c9af46_5b09_4b16_a585_3de8baa9d6b9.slice/crio-2c5d308e8ca49803b34012af7034bda7dca57f35d7b45f09bca00df1d6cc6aa9 WatchSource:0}: Error finding container 2c5d308e8ca49803b34012af7034bda7dca57f35d7b45f09bca00df1d6cc6aa9: Status 404 returned error can't find the container with id 2c5d308e8ca49803b34012af7034bda7dca57f35d7b45f09bca00df1d6cc6aa9
Jul 28 14:07:59.361516 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.361488    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"]
Jul 28 14:07:59.365260 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.365242    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.377238 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.377217    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"]
Jul 28 14:07:59.456811 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.456786    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gqg5\" (UniqueName: \"kubernetes.io/projected/792d6e64-1728-48ea-840f-228c68117c27-kube-api-access-6gqg5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.456916 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.456828    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.456916 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.456889    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.557659 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.557567    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.557823 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.557651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.557823 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.557810    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gqg5\" (UniqueName: \"kubernetes.io/projected/792d6e64-1728-48ea-840f-228c68117c27-kube-api-access-6gqg5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.558116 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.558011    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.558471 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.558452    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.568089 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.568063    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gqg5\" (UniqueName: \"kubernetes.io/projected/792d6e64-1728-48ea-840f-228c68117c27-kube-api-access-6gqg5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.727988 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.727965    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:07:59.759673 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.759646    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"]
Jul 28 14:07:59.764113 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.764088    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:07:59.776317 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.776290    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"]
Jul 28 14:07:59.800384 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.800351    2579 generic.go:356] "Generic (PLEG): container finished" podID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerID="563c60c19c26948767b873670943c8a1f93554978e6176846e08c46f955985fc" exitCode=0
Jul 28 14:07:59.800574 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.800392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c" event={"ID":"1ca6eeb4-4269-4ec0-a231-da61c5816719","Type":"ContainerDied","Data":"563c60c19c26948767b873670943c8a1f93554978e6176846e08c46f955985fc"}
Jul 28 14:07:59.802377 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.802356    2579 generic.go:356] "Generic (PLEG): container finished" podID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerID="8db8cc2e79d0022a4ab1645fa1c77808a3f5bc7d82397cfd2fd79395b2197e35" exitCode=0
Jul 28 14:07:59.802471 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.802435    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt" event={"ID":"54c9af46-5b09-4b16-a585-3de8baa9d6b9","Type":"ContainerDied","Data":"8db8cc2e79d0022a4ab1645fa1c77808a3f5bc7d82397cfd2fd79395b2197e35"}
Jul 28 14:07:59.802523 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.802473    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt" event={"ID":"54c9af46-5b09-4b16-a585-3de8baa9d6b9","Type":"ContainerStarted","Data":"2c5d308e8ca49803b34012af7034bda7dca57f35d7b45f09bca00df1d6cc6aa9"}
Jul 28 14:07:59.859383 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.859358    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:07:59.859511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.859411    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:07:59.859511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.859433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpwl7\" (UniqueName: \"kubernetes.io/projected/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-kube-api-access-jpwl7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:07:59.883302 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.883280    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"]
Jul 28 14:07:59.886114 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:07:59.886092    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792d6e64_1728_48ea_840f_228c68117c27.slice/crio-0ac94c356350252e9c46ced22652ac8977e2edfac0da503449291bee4d330bb2 WatchSource:0}: Error finding container 0ac94c356350252e9c46ced22652ac8977e2edfac0da503449291bee4d330bb2: Status 404 returned error can't find the container with id 0ac94c356350252e9c46ced22652ac8977e2edfac0da503449291bee4d330bb2
Jul 28 14:07:59.960381 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.960355    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:07:59.960480 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.960396    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpwl7\" (UniqueName: \"kubernetes.io/projected/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-kube-api-access-jpwl7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:07:59.960480 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.960465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:07:59.960772 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.960750    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:07:59.960837 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.960805    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:07:59.971211 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:07:59.971182    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpwl7\" (UniqueName: \"kubernetes.io/projected/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-kube-api-access-jpwl7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:08:00.116121 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.116044    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:08:00.262749 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.262711    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"]
Jul 28 14:08:00.265216 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:08:00.265190    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1cf73ab_75f2_4779_b3da_2a0f653ddb40.slice/crio-2ebeaabcf78ab9d085be1697a008254b6653b279db217a8d1c39adc5ae85998b WatchSource:0}: Error finding container 2ebeaabcf78ab9d085be1697a008254b6653b279db217a8d1c39adc5ae85998b: Status 404 returned error can't find the container with id 2ebeaabcf78ab9d085be1697a008254b6653b279db217a8d1c39adc5ae85998b
Jul 28 14:08:00.809772 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.809736    2579 generic.go:356] "Generic (PLEG): container finished" podID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerID="ccd9ca898cde4b694dd3ffa4e25bb490795762aa75a998fdd841cc791149fefe" exitCode=0
Jul 28 14:08:00.809904 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.809832    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5" event={"ID":"c1cf73ab-75f2-4779-b3da-2a0f653ddb40","Type":"ContainerDied","Data":"ccd9ca898cde4b694dd3ffa4e25bb490795762aa75a998fdd841cc791149fefe"}
Jul 28 14:08:00.809904 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.809886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5" event={"ID":"c1cf73ab-75f2-4779-b3da-2a0f653ddb40","Type":"ContainerStarted","Data":"2ebeaabcf78ab9d085be1697a008254b6653b279db217a8d1c39adc5ae85998b"}
Jul 28 14:08:00.811530 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.811506    2579 generic.go:356] "Generic (PLEG): container finished" podID="792d6e64-1728-48ea-840f-228c68117c27" containerID="654cf91507e5bebf5d892678e45edfbaabe5c3bd6d9b81ee611859b10235b202" exitCode=0
Jul 28 14:08:00.811632 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.811603    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c" event={"ID":"792d6e64-1728-48ea-840f-228c68117c27","Type":"ContainerDied","Data":"654cf91507e5bebf5d892678e45edfbaabe5c3bd6d9b81ee611859b10235b202"}
Jul 28 14:08:00.811703 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.811648    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c" event={"ID":"792d6e64-1728-48ea-840f-228c68117c27","Type":"ContainerStarted","Data":"0ac94c356350252e9c46ced22652ac8977e2edfac0da503449291bee4d330bb2"}
Jul 28 14:08:00.813939 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.813918    2579 generic.go:356] "Generic (PLEG): container finished" podID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerID="c383d1b4fb2050bcc1ab945a4491588521b2c9fb50289036f41ac6a02c3f2a65" exitCode=0
Jul 28 14:08:00.814063 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.813984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c" event={"ID":"1ca6eeb4-4269-4ec0-a231-da61c5816719","Type":"ContainerDied","Data":"c383d1b4fb2050bcc1ab945a4491588521b2c9fb50289036f41ac6a02c3f2a65"}
Jul 28 14:08:00.816342 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.816316    2579 generic.go:356] "Generic (PLEG): container finished" podID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerID="db1f6b7edc1163af0a4125c4e624f4dd3047af186744eb936a341418e8654951" exitCode=0
Jul 28 14:08:00.816429 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:00.816406    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt" event={"ID":"54c9af46-5b09-4b16-a585-3de8baa9d6b9","Type":"ContainerDied","Data":"db1f6b7edc1163af0a4125c4e624f4dd3047af186744eb936a341418e8654951"}
Jul 28 14:08:01.823773 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:01.823743    2579 generic.go:356] "Generic (PLEG): container finished" podID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerID="4c03bf4aa232f0434016c55a8c5ed85af1e3cd8b3086468602d92af328dbe1b2" exitCode=0
Jul 28 14:08:01.824252 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:01.823824    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5" event={"ID":"c1cf73ab-75f2-4779-b3da-2a0f653ddb40","Type":"ContainerDied","Data":"4c03bf4aa232f0434016c55a8c5ed85af1e3cd8b3086468602d92af328dbe1b2"}
Jul 28 14:08:01.826486 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:01.826463    2579 generic.go:356] "Generic (PLEG): container finished" podID="792d6e64-1728-48ea-840f-228c68117c27" containerID="23e657b06963362b7502e299c4eee1c58c8642ad88d7d08a55762fe3c89479c7" exitCode=0
Jul 28 14:08:01.826601 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:01.826548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c" event={"ID":"792d6e64-1728-48ea-840f-228c68117c27","Type":"ContainerDied","Data":"23e657b06963362b7502e299c4eee1c58c8642ad88d7d08a55762fe3c89479c7"}
Jul 28 14:08:01.829209 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:01.829187    2579 generic.go:356] "Generic (PLEG): container finished" podID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerID="ea88e8e2386b76c6a793a700b9a1f31e78e9607963c45220b626f75842ab22c1" exitCode=0
Jul 28 14:08:01.829305 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:01.829264    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt" event={"ID":"54c9af46-5b09-4b16-a585-3de8baa9d6b9","Type":"ContainerDied","Data":"ea88e8e2386b76c6a793a700b9a1f31e78e9607963c45220b626f75842ab22c1"}
Jul 28 14:08:01.964153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:01.964128    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:08:02.081255 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.081229    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-bundle\") pod \"1ca6eeb4-4269-4ec0-a231-da61c5816719\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") "
Jul 28 14:08:02.081397 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.081292    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-util\") pod \"1ca6eeb4-4269-4ec0-a231-da61c5816719\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") "
Jul 28 14:08:02.081397 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.081321    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsjk5\" (UniqueName: \"kubernetes.io/projected/1ca6eeb4-4269-4ec0-a231-da61c5816719-kube-api-access-lsjk5\") pod \"1ca6eeb4-4269-4ec0-a231-da61c5816719\" (UID: \"1ca6eeb4-4269-4ec0-a231-da61c5816719\") "
Jul 28 14:08:02.081695 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.081665    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-bundle" (OuterVolumeSpecName: "bundle") pod "1ca6eeb4-4269-4ec0-a231-da61c5816719" (UID: "1ca6eeb4-4269-4ec0-a231-da61c5816719"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:08:02.083521 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.083501    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ca6eeb4-4269-4ec0-a231-da61c5816719-kube-api-access-lsjk5" (OuterVolumeSpecName: "kube-api-access-lsjk5") pod "1ca6eeb4-4269-4ec0-a231-da61c5816719" (UID: "1ca6eeb4-4269-4ec0-a231-da61c5816719"). InnerVolumeSpecName "kube-api-access-lsjk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:08:02.087179 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.087158    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-util" (OuterVolumeSpecName: "util") pod "1ca6eeb4-4269-4ec0-a231-da61c5816719" (UID: "1ca6eeb4-4269-4ec0-a231-da61c5816719"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:08:02.182783 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.182758    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-util\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:02.182783 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.182782    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsjk5\" (UniqueName: \"kubernetes.io/projected/1ca6eeb4-4269-4ec0-a231-da61c5816719-kube-api-access-lsjk5\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:02.182926 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.182792    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ca6eeb4-4269-4ec0-a231-da61c5816719-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:02.836726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.836698    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c"
Jul 28 14:08:02.836726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.836718    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c" event={"ID":"1ca6eeb4-4269-4ec0-a231-da61c5816719","Type":"ContainerDied","Data":"0513290dbc958509528270a67fd85b79660886ac3739e47e89931871ebaea268"}
Jul 28 14:08:02.837298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.836768    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0513290dbc958509528270a67fd85b79660886ac3739e47e89931871ebaea268"
Jul 28 14:08:02.839190 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.839162    2579 generic.go:356] "Generic (PLEG): container finished" podID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerID="a31ece726761f5df84a7f4fbcb31c28aa1eaa4c07681c7152cbf25583e8a61a8" exitCode=0
Jul 28 14:08:02.839591 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.839245    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5" event={"ID":"c1cf73ab-75f2-4779-b3da-2a0f653ddb40","Type":"ContainerDied","Data":"a31ece726761f5df84a7f4fbcb31c28aa1eaa4c07681c7152cbf25583e8a61a8"}
Jul 28 14:08:02.841664 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.841641    2579 generic.go:356] "Generic (PLEG): container finished" podID="792d6e64-1728-48ea-840f-228c68117c27" containerID="c87fbb7d9d269a0b2be8e29fd362782ab5bc204c27f8c04544612207b7817557" exitCode=0
Jul 28 14:08:02.841769 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.841715    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c" event={"ID":"792d6e64-1728-48ea-840f-228c68117c27","Type":"ContainerDied","Data":"c87fbb7d9d269a0b2be8e29fd362782ab5bc204c27f8c04544612207b7817557"}
Jul 28 14:08:02.981736 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:02.981710    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:08:03.091330 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.091241    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-bundle\") pod \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") "
Jul 28 14:08:03.091330 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.091281    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-util\") pod \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") "
Jul 28 14:08:03.091561 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.091388    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv7qh\" (UniqueName: \"kubernetes.io/projected/54c9af46-5b09-4b16-a585-3de8baa9d6b9-kube-api-access-bv7qh\") pod \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\" (UID: \"54c9af46-5b09-4b16-a585-3de8baa9d6b9\") "
Jul 28 14:08:03.091855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.091825    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-bundle" (OuterVolumeSpecName: "bundle") pod "54c9af46-5b09-4b16-a585-3de8baa9d6b9" (UID: "54c9af46-5b09-4b16-a585-3de8baa9d6b9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:08:03.093659 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.093633    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54c9af46-5b09-4b16-a585-3de8baa9d6b9-kube-api-access-bv7qh" (OuterVolumeSpecName: "kube-api-access-bv7qh") pod "54c9af46-5b09-4b16-a585-3de8baa9d6b9" (UID: "54c9af46-5b09-4b16-a585-3de8baa9d6b9"). InnerVolumeSpecName "kube-api-access-bv7qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:08:03.096721 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.096701    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-util" (OuterVolumeSpecName: "util") pod "54c9af46-5b09-4b16-a585-3de8baa9d6b9" (UID: "54c9af46-5b09-4b16-a585-3de8baa9d6b9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:08:03.192246 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.192211    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv7qh\" (UniqueName: \"kubernetes.io/projected/54c9af46-5b09-4b16-a585-3de8baa9d6b9-kube-api-access-bv7qh\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:03.192246 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.192239    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:03.192246 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.192248    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c9af46-5b09-4b16-a585-3de8baa9d6b9-util\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:03.849119 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.849089    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt"
Jul 28 14:08:03.849593 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.849121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt" event={"ID":"54c9af46-5b09-4b16-a585-3de8baa9d6b9","Type":"ContainerDied","Data":"2c5d308e8ca49803b34012af7034bda7dca57f35d7b45f09bca00df1d6cc6aa9"}
Jul 28 14:08:03.849593 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.849169    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c5d308e8ca49803b34012af7034bda7dca57f35d7b45f09bca00df1d6cc6aa9"
Jul 28 14:08:03.985729 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:03.985707    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:08:04.019406 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.019379    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:08:04.100879 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.100785    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-util\") pod \"792d6e64-1728-48ea-840f-228c68117c27\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") "
Jul 28 14:08:04.100879 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.100860    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gqg5\" (UniqueName: \"kubernetes.io/projected/792d6e64-1728-48ea-840f-228c68117c27-kube-api-access-6gqg5\") pod \"792d6e64-1728-48ea-840f-228c68117c27\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") "
Jul 28 14:08:04.101153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.100888    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-bundle\") pod \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") "
Jul 28 14:08:04.101153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.100915    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-util\") pod \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") "
Jul 28 14:08:04.101153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.100937    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpwl7\" (UniqueName: \"kubernetes.io/projected/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-kube-api-access-jpwl7\") pod \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\" (UID: \"c1cf73ab-75f2-4779-b3da-2a0f653ddb40\") "
Jul 28 14:08:04.101153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.100966    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-bundle\") pod \"792d6e64-1728-48ea-840f-228c68117c27\" (UID: \"792d6e64-1728-48ea-840f-228c68117c27\") "
Jul 28 14:08:04.101823 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.101534    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-bundle" (OuterVolumeSpecName: "bundle") pod "792d6e64-1728-48ea-840f-228c68117c27" (UID: "792d6e64-1728-48ea-840f-228c68117c27"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:08:04.101823 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.101597    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-bundle" (OuterVolumeSpecName: "bundle") pod "c1cf73ab-75f2-4779-b3da-2a0f653ddb40" (UID: "c1cf73ab-75f2-4779-b3da-2a0f653ddb40"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:08:04.103341 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.103317    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-kube-api-access-jpwl7" (OuterVolumeSpecName: "kube-api-access-jpwl7") pod "c1cf73ab-75f2-4779-b3da-2a0f653ddb40" (UID: "c1cf73ab-75f2-4779-b3da-2a0f653ddb40"). InnerVolumeSpecName "kube-api-access-jpwl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:08:04.103341 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.103329    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/792d6e64-1728-48ea-840f-228c68117c27-kube-api-access-6gqg5" (OuterVolumeSpecName: "kube-api-access-6gqg5") pod "792d6e64-1728-48ea-840f-228c68117c27" (UID: "792d6e64-1728-48ea-840f-228c68117c27"). InnerVolumeSpecName "kube-api-access-6gqg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:08:04.106777 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.106749    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-util" (OuterVolumeSpecName: "util") pod "792d6e64-1728-48ea-840f-228c68117c27" (UID: "792d6e64-1728-48ea-840f-228c68117c27"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:08:04.106955 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.106937    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-util" (OuterVolumeSpecName: "util") pod "c1cf73ab-75f2-4779-b3da-2a0f653ddb40" (UID: "c1cf73ab-75f2-4779-b3da-2a0f653ddb40"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:08:04.201781 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.201742    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gqg5\" (UniqueName: \"kubernetes.io/projected/792d6e64-1728-48ea-840f-228c68117c27-kube-api-access-6gqg5\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:04.201781 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.201774    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:04.201781 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.201784    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-util\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:04.201781 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.201793    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpwl7\" (UniqueName: \"kubernetes.io/projected/c1cf73ab-75f2-4779-b3da-2a0f653ddb40-kube-api-access-jpwl7\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:04.202082 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.201801    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:04.202082 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.201809    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792d6e64-1728-48ea-840f-228c68117c27-util\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:04.856390 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.856361    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c"
Jul 28 14:08:04.856837 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.856398    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c" event={"ID":"792d6e64-1728-48ea-840f-228c68117c27","Type":"ContainerDied","Data":"0ac94c356350252e9c46ced22652ac8977e2edfac0da503449291bee4d330bb2"}
Jul 28 14:08:04.856837 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.856483    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ac94c356350252e9c46ced22652ac8977e2edfac0da503449291bee4d330bb2"
Jul 28 14:08:04.858557 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.858530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5" event={"ID":"c1cf73ab-75f2-4779-b3da-2a0f653ddb40","Type":"ContainerDied","Data":"2ebeaabcf78ab9d085be1697a008254b6653b279db217a8d1c39adc5ae85998b"}
Jul 28 14:08:04.858691 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.858574    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ebeaabcf78ab9d085be1697a008254b6653b279db217a8d1c39adc5ae85998b"
Jul 28 14:08:04.858843 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:04.858820    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5"
Jul 28 14:08:08.438587 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:08.438548    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fdb4fcdfd-cmwjq"]
Jul 28 14:08:29.721476 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721441    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"]
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721800    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerName="util"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721811    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerName="util"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721819    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerName="pull"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721824    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerName="pull"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721835    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerName="extract"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721840    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerName="extract"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721847    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792d6e64-1728-48ea-840f-228c68117c27" containerName="pull"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721852    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="792d6e64-1728-48ea-840f-228c68117c27" containerName="pull"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721860    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerName="pull"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721864    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerName="pull"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721870    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerName="extract"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721875    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerName="extract"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721882    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerName="util"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721888    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerName="util"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721897    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerName="util"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721901    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerName="util"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721906    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerName="extract"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721910    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerName="extract"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721917    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792d6e64-1728-48ea-840f-228c68117c27" containerName="util"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721922    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="792d6e64-1728-48ea-840f-228c68117c27" containerName="util"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721927    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerName="pull"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721931    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerName="pull"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721936    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792d6e64-1728-48ea-840f-228c68117c27" containerName="extract"
Jul 28 14:08:29.721966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.721940    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="792d6e64-1728-48ea-840f-228c68117c27" containerName="extract"
Jul 28 14:08:29.722805 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.722044    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ca6eeb4-4269-4ec0-a231-da61c5816719" containerName="extract"
Jul 28 14:08:29.722805 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.722053    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c9af46-5b09-4b16-a585-3de8baa9d6b9" containerName="extract"
Jul 28 14:08:29.722805 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.722059    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="792d6e64-1728-48ea-840f-228c68117c27" containerName="extract"
Jul 28 14:08:29.722805 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.722064    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1cf73ab-75f2-4779-b3da-2a0f653ddb40" containerName="extract"
Jul 28 14:08:29.724720 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.724702    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:29.728289 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.728267    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 14:08:29.728398 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.728361    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 14:08:29.728767 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.728753    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 14:08:29.728980 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.728965    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wwwfv\""
Jul 28 14:08:29.729276 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.729253    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 14:08:29.737367 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.737346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"]
Jul 28 14:08:29.828310 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.828279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/89d3b87e-94de-4b39-a9a3-0b309bc02fd5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5xx2m\" (UID: \"89d3b87e-94de-4b39-a9a3-0b309bc02fd5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:29.828507 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.828329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkfpb\" (UniqueName: \"kubernetes.io/projected/89d3b87e-94de-4b39-a9a3-0b309bc02fd5-kube-api-access-dkfpb\") pod \"kuadrant-console-plugin-66b5774985-5xx2m\" (UID: \"89d3b87e-94de-4b39-a9a3-0b309bc02fd5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:29.828507 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.828350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/89d3b87e-94de-4b39-a9a3-0b309bc02fd5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5xx2m\" (UID: \"89d3b87e-94de-4b39-a9a3-0b309bc02fd5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:29.929262 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.929226    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/89d3b87e-94de-4b39-a9a3-0b309bc02fd5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5xx2m\" (UID: \"89d3b87e-94de-4b39-a9a3-0b309bc02fd5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:29.929414 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.929276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkfpb\" (UniqueName: \"kubernetes.io/projected/89d3b87e-94de-4b39-a9a3-0b309bc02fd5-kube-api-access-dkfpb\") pod \"kuadrant-console-plugin-66b5774985-5xx2m\" (UID: \"89d3b87e-94de-4b39-a9a3-0b309bc02fd5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:29.929414 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.929391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/89d3b87e-94de-4b39-a9a3-0b309bc02fd5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5xx2m\" (UID: \"89d3b87e-94de-4b39-a9a3-0b309bc02fd5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:29.929879 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.929859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/89d3b87e-94de-4b39-a9a3-0b309bc02fd5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5xx2m\" (UID: \"89d3b87e-94de-4b39-a9a3-0b309bc02fd5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:29.931971 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.931950    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/89d3b87e-94de-4b39-a9a3-0b309bc02fd5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5xx2m\" (UID: \"89d3b87e-94de-4b39-a9a3-0b309bc02fd5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:29.939206 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:29.939180    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkfpb\" (UniqueName: \"kubernetes.io/projected/89d3b87e-94de-4b39-a9a3-0b309bc02fd5-kube-api-access-dkfpb\") pod \"kuadrant-console-plugin-66b5774985-5xx2m\" (UID: \"89d3b87e-94de-4b39-a9a3-0b309bc02fd5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:30.034092 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:30.034061    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"
Jul 28 14:08:30.187788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:30.187758    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m"]
Jul 28 14:08:30.191073 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:08:30.191045    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89d3b87e_94de_4b39_a9a3_0b309bc02fd5.slice/crio-52e9025b3f967c86e85898c253d286d471bc7396c44c2b60ab8ee57bb0734791 WatchSource:0}: Error finding container 52e9025b3f967c86e85898c253d286d471bc7396c44c2b60ab8ee57bb0734791: Status 404 returned error can't find the container with id 52e9025b3f967c86e85898c253d286d471bc7396c44c2b60ab8ee57bb0734791
Jul 28 14:08:30.996778 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:30.996732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m" event={"ID":"89d3b87e-94de-4b39-a9a3-0b309bc02fd5","Type":"ContainerStarted","Data":"52e9025b3f967c86e85898c253d286d471bc7396c44c2b60ab8ee57bb0734791"}
Jul 28 14:08:33.463729 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.463671    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fdb4fcdfd-cmwjq" podUID="9935df18-a3cc-4617-bb38-66678577db20" containerName="console" containerID="cri-o://9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80" gracePeriod=15
Jul 28 14:08:33.762048 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.761995    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:08:33.866653 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.866618    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-service-ca\") pod \"9935df18-a3cc-4617-bb38-66678577db20\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") "
Jul 28 14:08:33.866653 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.866660    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-trusted-ca-bundle\") pod \"9935df18-a3cc-4617-bb38-66678577db20\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") "
Jul 28 14:08:33.866893 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.866699    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-oauth-config\") pod \"9935df18-a3cc-4617-bb38-66678577db20\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") "
Jul 28 14:08:33.866893 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.866743    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67wh4\" (UniqueName: \"kubernetes.io/projected/9935df18-a3cc-4617-bb38-66678577db20-kube-api-access-67wh4\") pod \"9935df18-a3cc-4617-bb38-66678577db20\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") "
Jul 28 14:08:33.866893 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.866773    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-oauth-serving-cert\") pod \"9935df18-a3cc-4617-bb38-66678577db20\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") "
Jul 28 14:08:33.866893 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.866803    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-console-config\") pod \"9935df18-a3cc-4617-bb38-66678577db20\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") "
Jul 28 14:08:33.866893 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.866837    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-serving-cert\") pod \"9935df18-a3cc-4617-bb38-66678577db20\" (UID: \"9935df18-a3cc-4617-bb38-66678577db20\") "
Jul 28 14:08:33.867255 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.867054    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-service-ca" (OuterVolumeSpecName: "service-ca") pod "9935df18-a3cc-4617-bb38-66678577db20" (UID: "9935df18-a3cc-4617-bb38-66678577db20"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:08:33.867353 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.867293    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:33.867485 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.867207    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9935df18-a3cc-4617-bb38-66678577db20" (UID: "9935df18-a3cc-4617-bb38-66678577db20"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:08:33.867578 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.867532    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-console-config" (OuterVolumeSpecName: "console-config") pod "9935df18-a3cc-4617-bb38-66678577db20" (UID: "9935df18-a3cc-4617-bb38-66678577db20"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:08:33.867670 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.867651    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9935df18-a3cc-4617-bb38-66678577db20" (UID: "9935df18-a3cc-4617-bb38-66678577db20"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:08:33.869358 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.869337    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9935df18-a3cc-4617-bb38-66678577db20" (UID: "9935df18-a3cc-4617-bb38-66678577db20"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:08:33.869670 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.869653    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9935df18-a3cc-4617-bb38-66678577db20" (UID: "9935df18-a3cc-4617-bb38-66678577db20"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:08:33.869736 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.869683    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9935df18-a3cc-4617-bb38-66678577db20-kube-api-access-67wh4" (OuterVolumeSpecName: "kube-api-access-67wh4") pod "9935df18-a3cc-4617-bb38-66678577db20" (UID: "9935df18-a3cc-4617-bb38-66678577db20"). InnerVolumeSpecName "kube-api-access-67wh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:08:33.968275 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.968236    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-oauth-serving-cert\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:33.968275 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.968273    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-console-config\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:33.968531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.968289    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-serving-cert\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:33.968531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.968305    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9935df18-a3cc-4617-bb38-66678577db20-trusted-ca-bundle\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:33.968531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.968318    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9935df18-a3cc-4617-bb38-66678577db20-console-oauth-config\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:33.968531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:33.968334    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67wh4\" (UniqueName: \"kubernetes.io/projected/9935df18-a3cc-4617-bb38-66678577db20-kube-api-access-67wh4\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:08:34.022002 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.021961    2579 generic.go:356] "Generic (PLEG): container finished" podID="9935df18-a3cc-4617-bb38-66678577db20" containerID="9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80" exitCode=0
Jul 28 14:08:34.022211 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.022071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb4fcdfd-cmwjq" event={"ID":"9935df18-a3cc-4617-bb38-66678577db20","Type":"ContainerDied","Data":"9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80"}
Jul 28 14:08:34.022211 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.022160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb4fcdfd-cmwjq" event={"ID":"9935df18-a3cc-4617-bb38-66678577db20","Type":"ContainerDied","Data":"35e8a6fd17ff327c6968e0aa7eff541b5b12ddf5306c81cb12cbcf7ea8fdf8af"}
Jul 28 14:08:34.022211 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.022191    2579 scope.go:117] "RemoveContainer" containerID="9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80"
Jul 28 14:08:34.022410 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.022393    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb4fcdfd-cmwjq"
Jul 28 14:08:34.034909 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.034884    2579 scope.go:117] "RemoveContainer" containerID="9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80"
Jul 28 14:08:34.035254 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:08:34.035227    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80\": container with ID starting with 9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80 not found: ID does not exist" containerID="9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80"
Jul 28 14:08:34.035359 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.035266    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80"} err="failed to get container status \"9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80\": rpc error: code = NotFound desc = could not find container \"9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80\": container with ID starting with 9190e4a4d2b23952d00a88a6778381630a5bc43d6feb294346533f59ba273b80 not found: ID does not exist"
Jul 28 14:08:34.056576 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.056554    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fdb4fcdfd-cmwjq"]
Jul 28 14:08:34.068680 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.068658    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fdb4fcdfd-cmwjq"]
Jul 28 14:08:34.348980 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:34.348906    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9935df18-a3cc-4617-bb38-66678577db20" path="/var/lib/kubelet/pods/9935df18-a3cc-4617-bb38-66678577db20/volumes"
Jul 28 14:08:53.141306 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:53.141268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m" event={"ID":"89d3b87e-94de-4b39-a9a3-0b309bc02fd5","Type":"ContainerStarted","Data":"426dff39ee2180f26fb712a851abcaabc2783112ac7259e7f5cca81f0e328716"}
Jul 28 14:08:53.171461 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:08:53.171405    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xx2m" podStartSLOduration=2.193387366 podStartE2EDuration="24.171389592s" totalImagesPullingTime="21.978002226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:08:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:08:53.16951928 +0000 UTC m=+549.296909973" watchObservedRunningTime="2026-07-28 14:08:53.171389592 +0000 UTC m=+549.298780283"
Jul 28 14:09:22.986712 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:22.986664    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:09:22.987239 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:22.987222    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9935df18-a3cc-4617-bb38-66678577db20" containerName="console"
Jul 28 14:09:22.987287 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:22.987241    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9935df18-a3cc-4617-bb38-66678577db20" containerName="console"
Jul 28 14:09:22.987326 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:22.987302    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9935df18-a3cc-4617-bb38-66678577db20" containerName="console"
Jul 28 14:09:23.009926 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.009833    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:09:23.010123 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.009931    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:23.012491 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.012470    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 14:09:23.023588 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.023560    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:09:23.116731 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.116688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/015b1470-ed09-4093-8e1a-373d705b7061-config-file\") pod \"limitador-limitador-54d6d988d5-zrc7c\" (UID: \"015b1470-ed09-4093-8e1a-373d705b7061\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:23.116908 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.116765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz2bx\" (UniqueName: \"kubernetes.io/projected/015b1470-ed09-4093-8e1a-373d705b7061-kube-api-access-dz2bx\") pod \"limitador-limitador-54d6d988d5-zrc7c\" (UID: \"015b1470-ed09-4093-8e1a-373d705b7061\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:23.217914 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.217873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz2bx\" (UniqueName: \"kubernetes.io/projected/015b1470-ed09-4093-8e1a-373d705b7061-kube-api-access-dz2bx\") pod \"limitador-limitador-54d6d988d5-zrc7c\" (UID: \"015b1470-ed09-4093-8e1a-373d705b7061\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:23.218141 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.217984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/015b1470-ed09-4093-8e1a-373d705b7061-config-file\") pod \"limitador-limitador-54d6d988d5-zrc7c\" (UID: \"015b1470-ed09-4093-8e1a-373d705b7061\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:23.218652 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.218632    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/015b1470-ed09-4093-8e1a-373d705b7061-config-file\") pod \"limitador-limitador-54d6d988d5-zrc7c\" (UID: \"015b1470-ed09-4093-8e1a-373d705b7061\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:23.227865 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.227838    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz2bx\" (UniqueName: \"kubernetes.io/projected/015b1470-ed09-4093-8e1a-373d705b7061-kube-api-access-dz2bx\") pod \"limitador-limitador-54d6d988d5-zrc7c\" (UID: \"015b1470-ed09-4093-8e1a-373d705b7061\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:23.320255 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.320167    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:23.406763 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.403000    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-k6w95"]
Jul 28 14:09:23.408493 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.408465    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k6w95"
Jul 28 14:09:23.412298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.412269    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hnlqr\""
Jul 28 14:09:23.418487 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.418456    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k6w95"]
Jul 28 14:09:23.489438 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.489411    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:09:23.491059 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:09:23.491007    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod015b1470_ed09_4093_8e1a_373d705b7061.slice/crio-297b8584fe6e387654a3537d592d3c9c35a4f1d9da69ce32a5b3a139a39d2d11 WatchSource:0}: Error finding container 297b8584fe6e387654a3537d592d3c9c35a4f1d9da69ce32a5b3a139a39d2d11: Status 404 returned error can't find the container with id 297b8584fe6e387654a3537d592d3c9c35a4f1d9da69ce32a5b3a139a39d2d11
Jul 28 14:09:23.521262 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.521230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjr7l\" (UniqueName: \"kubernetes.io/projected/4b64d2dd-21f5-4846-9d06-e3522549120a-kube-api-access-hjr7l\") pod \"authorino-58cd955675-k6w95\" (UID: \"4b64d2dd-21f5-4846-9d06-e3522549120a\") " pod="kuadrant-system/authorino-58cd955675-k6w95"
Jul 28 14:09:23.621711 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.621629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjr7l\" (UniqueName: \"kubernetes.io/projected/4b64d2dd-21f5-4846-9d06-e3522549120a-kube-api-access-hjr7l\") pod \"authorino-58cd955675-k6w95\" (UID: \"4b64d2dd-21f5-4846-9d06-e3522549120a\") " pod="kuadrant-system/authorino-58cd955675-k6w95"
Jul 28 14:09:23.638119 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.638092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjr7l\" (UniqueName: \"kubernetes.io/projected/4b64d2dd-21f5-4846-9d06-e3522549120a-kube-api-access-hjr7l\") pod \"authorino-58cd955675-k6w95\" (UID: \"4b64d2dd-21f5-4846-9d06-e3522549120a\") " pod="kuadrant-system/authorino-58cd955675-k6w95"
Jul 28 14:09:23.720206 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.720169    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k6w95"
Jul 28 14:09:23.868469 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:23.868445    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k6w95"]
Jul 28 14:09:23.870296 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:09:23.870272    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b64d2dd_21f5_4846_9d06_e3522549120a.slice/crio-d03fe9e61447cfb71b4f81aae11519319c149e33cbd3d2c780f837805347157b WatchSource:0}: Error finding container d03fe9e61447cfb71b4f81aae11519319c149e33cbd3d2c780f837805347157b: Status 404 returned error can't find the container with id d03fe9e61447cfb71b4f81aae11519319c149e33cbd3d2c780f837805347157b
Jul 28 14:09:24.309856 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:24.309770    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c" event={"ID":"015b1470-ed09-4093-8e1a-373d705b7061","Type":"ContainerStarted","Data":"297b8584fe6e387654a3537d592d3c9c35a4f1d9da69ce32a5b3a139a39d2d11"}
Jul 28 14:09:24.311741 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:24.311693    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k6w95" event={"ID":"4b64d2dd-21f5-4846-9d06-e3522549120a","Type":"ContainerStarted","Data":"d03fe9e61447cfb71b4f81aae11519319c149e33cbd3d2c780f837805347157b"}
Jul 28 14:09:28.343218 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:28.343120    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c" event={"ID":"015b1470-ed09-4093-8e1a-373d705b7061","Type":"ContainerStarted","Data":"c79de95863cb1f1ded27a05b83f835dce1122aee4a38661e6d39d53f9d263497"}
Jul 28 14:09:28.343619 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:28.343256    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:28.344629 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:28.344604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k6w95" event={"ID":"4b64d2dd-21f5-4846-9d06-e3522549120a","Type":"ContainerStarted","Data":"4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9"}
Jul 28 14:09:28.374926 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:28.374872    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c" podStartSLOduration=1.827322895 podStartE2EDuration="6.374854714s" totalImagesPullingTime="4.547531819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:09:28.3743055 +0000 UTC m=+584.501696191" watchObservedRunningTime="2026-07-28 14:09:28.374854714 +0000 UTC m=+584.502245405"
Jul 28 14:09:28.395230 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:28.395185    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-k6w95" podStartSLOduration=1.219892002 podStartE2EDuration="5.395172626s" totalImagesPullingTime="4.175280624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:09:28.392895812 +0000 UTC m=+584.520286500" watchObservedRunningTime="2026-07-28 14:09:28.395172626 +0000 UTC m=+584.522563317"
Jul 28 14:09:28.509694 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:28.509657    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k6w95"]
Jul 28 14:09:30.355481 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:30.355444    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-k6w95" podUID="4b64d2dd-21f5-4846-9d06-e3522549120a" containerName="authorino" containerID="cri-o://4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9" gracePeriod=30
Jul 28 14:09:30.609202 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:30.609136    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k6w95"
Jul 28 14:09:30.692857 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:30.692820    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjr7l\" (UniqueName: \"kubernetes.io/projected/4b64d2dd-21f5-4846-9d06-e3522549120a-kube-api-access-hjr7l\") pod \"4b64d2dd-21f5-4846-9d06-e3522549120a\" (UID: \"4b64d2dd-21f5-4846-9d06-e3522549120a\") "
Jul 28 14:09:30.694999 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:30.694969    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b64d2dd-21f5-4846-9d06-e3522549120a-kube-api-access-hjr7l" (OuterVolumeSpecName: "kube-api-access-hjr7l") pod "4b64d2dd-21f5-4846-9d06-e3522549120a" (UID: "4b64d2dd-21f5-4846-9d06-e3522549120a"). InnerVolumeSpecName "kube-api-access-hjr7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:09:30.793936 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:30.793899    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjr7l\" (UniqueName: \"kubernetes.io/projected/4b64d2dd-21f5-4846-9d06-e3522549120a-kube-api-access-hjr7l\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:09:31.361664 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:31.361628    2579 generic.go:356] "Generic (PLEG): container finished" podID="4b64d2dd-21f5-4846-9d06-e3522549120a" containerID="4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9" exitCode=0
Jul 28 14:09:31.362132 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:31.361677    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k6w95"
Jul 28 14:09:31.362132 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:31.361720    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k6w95" event={"ID":"4b64d2dd-21f5-4846-9d06-e3522549120a","Type":"ContainerDied","Data":"4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9"}
Jul 28 14:09:31.362132 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:31.361763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k6w95" event={"ID":"4b64d2dd-21f5-4846-9d06-e3522549120a","Type":"ContainerDied","Data":"d03fe9e61447cfb71b4f81aae11519319c149e33cbd3d2c780f837805347157b"}
Jul 28 14:09:31.362132 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:31.361783    2579 scope.go:117] "RemoveContainer" containerID="4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9"
Jul 28 14:09:31.371309 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:31.371292    2579 scope.go:117] "RemoveContainer" containerID="4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9"
Jul 28 14:09:31.371601 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:09:31.371580    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9\": container with ID starting with 4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9 not found: ID does not exist" containerID="4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9"
Jul 28 14:09:31.371673 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:31.371612    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9"} err="failed to get container status \"4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9\": rpc error: code = NotFound desc = could not find container \"4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9\": container with ID starting with 4d4d5f2ca5add0ad57fdc84848c5a064295a75551078db63d5a4ac161d2616e9 not found: ID does not exist"
Jul 28 14:09:31.390911 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:31.390884    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k6w95"]
Jul 28 14:09:31.397030 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:31.396992    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-k6w95"]
Jul 28 14:09:32.341538 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:32.341509    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b64d2dd-21f5-4846-9d06-e3522549120a" path="/var/lib/kubelet/pods/4b64d2dd-21f5-4846-9d06-e3522549120a/volumes"
Jul 28 14:09:39.351388 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:39.351348    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"
Jul 28 14:09:44.232086 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:44.232059    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:09:44.232494 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:09:44.232115    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:10:12.277981 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.277902    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7sq58"]
Jul 28 14:10:12.278454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.278342    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b64d2dd-21f5-4846-9d06-e3522549120a" containerName="authorino"
Jul 28 14:10:12.278454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.278357    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b64d2dd-21f5-4846-9d06-e3522549120a" containerName="authorino"
Jul 28 14:10:12.278454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.278421    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b64d2dd-21f5-4846-9d06-e3522549120a" containerName="authorino"
Jul 28 14:10:12.281364 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.281348    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7sq58"
Jul 28 14:10:12.284164 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.284134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hnlqr\""
Jul 28 14:10:12.296500 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.296479    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7sq58"]
Jul 28 14:10:12.353533 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.353499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dswqk\" (UniqueName: \"kubernetes.io/projected/9dae1b14-c8ff-4a17-a6d1-a4a7d592244a-kube-api-access-dswqk\") pod \"authorino-5d4b67c446-7sq58\" (UID: \"9dae1b14-c8ff-4a17-a6d1-a4a7d592244a\") " pod="kuadrant-system/authorino-5d4b67c446-7sq58"
Jul 28 14:10:12.454443 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.454398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dswqk\" (UniqueName: \"kubernetes.io/projected/9dae1b14-c8ff-4a17-a6d1-a4a7d592244a-kube-api-access-dswqk\") pod \"authorino-5d4b67c446-7sq58\" (UID: \"9dae1b14-c8ff-4a17-a6d1-a4a7d592244a\") " pod="kuadrant-system/authorino-5d4b67c446-7sq58"
Jul 28 14:10:12.470482 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.470457    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dswqk\" (UniqueName: \"kubernetes.io/projected/9dae1b14-c8ff-4a17-a6d1-a4a7d592244a-kube-api-access-dswqk\") pod \"authorino-5d4b67c446-7sq58\" (UID: \"9dae1b14-c8ff-4a17-a6d1-a4a7d592244a\") " pod="kuadrant-system/authorino-5d4b67c446-7sq58"
Jul 28 14:10:12.580270 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.580184    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7sq58"]
Jul 28 14:10:12.580458 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.580446    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7sq58"
Jul 28 14:10:12.616245 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.616216    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-775959b5c9-g7qx5"]
Jul 28 14:10:12.621180 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.621153    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-775959b5c9-g7qx5"
Jul 28 14:10:12.640557 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.640498    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-775959b5c9-g7qx5"]
Jul 28 14:10:12.656959 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.656504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgfvn\" (UniqueName: \"kubernetes.io/projected/1b945827-7a5d-4b16-96dd-219fcd66c07c-kube-api-access-pgfvn\") pod \"authorino-775959b5c9-g7qx5\" (UID: \"1b945827-7a5d-4b16-96dd-219fcd66c07c\") " pod="kuadrant-system/authorino-775959b5c9-g7qx5"
Jul 28 14:10:12.757494 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.757466    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgfvn\" (UniqueName: \"kubernetes.io/projected/1b945827-7a5d-4b16-96dd-219fcd66c07c-kube-api-access-pgfvn\") pod \"authorino-775959b5c9-g7qx5\" (UID: \"1b945827-7a5d-4b16-96dd-219fcd66c07c\") " pod="kuadrant-system/authorino-775959b5c9-g7qx5"
Jul 28 14:10:12.759093 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.759073    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7sq58"]
Jul 28 14:10:12.761172 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:10:12.761147    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9dae1b14_c8ff_4a17_a6d1_a4a7d592244a.slice/crio-97fe30d63d6a8588baf6075ff6251aab7cde733e31f4ff4e0e3b5f9d7f975675 WatchSource:0}: Error finding container 97fe30d63d6a8588baf6075ff6251aab7cde733e31f4ff4e0e3b5f9d7f975675: Status 404 returned error can't find the container with id 97fe30d63d6a8588baf6075ff6251aab7cde733e31f4ff4e0e3b5f9d7f975675
Jul 28 14:10:12.776166 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.776140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgfvn\" (UniqueName: \"kubernetes.io/projected/1b945827-7a5d-4b16-96dd-219fcd66c07c-kube-api-access-pgfvn\") pod \"authorino-775959b5c9-g7qx5\" (UID: \"1b945827-7a5d-4b16-96dd-219fcd66c07c\") " pod="kuadrant-system/authorino-775959b5c9-g7qx5"
Jul 28 14:10:12.824840 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.824810    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-775959b5c9-g7qx5"]
Jul 28 14:10:12.825098 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.825085    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-775959b5c9-g7qx5"
Jul 28 14:10:12.878879 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.877560    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6ddb96c995-st476"]
Jul 28 14:10:12.882974 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.882937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:10:12.886608 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.886577    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 14:10:12.904798 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.904769    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6ddb96c995-st476"]
Jul 28 14:10:12.959266 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.959239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km5pf\" (UniqueName: \"kubernetes.io/projected/78cd0f77-4dde-42aa-960e-f88aa51127f6-kube-api-access-km5pf\") pod \"authorino-6ddb96c995-st476\" (UID: \"78cd0f77-4dde-42aa-960e-f88aa51127f6\") " pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:10:12.959423 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.959294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78cd0f77-4dde-42aa-960e-f88aa51127f6-tls-cert\") pod \"authorino-6ddb96c995-st476\" (UID: \"78cd0f77-4dde-42aa-960e-f88aa51127f6\") " pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:10:12.970402 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:12.970381    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-775959b5c9-g7qx5"]
Jul 28 14:10:12.972697 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:10:12.972668    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b945827_7a5d_4b16_96dd_219fcd66c07c.slice/crio-b5612e14998850453c591a731c3416a939a8194d0cce1ee7c692ee4bfb96b700 WatchSource:0}: Error finding container b5612e14998850453c591a731c3416a939a8194d0cce1ee7c692ee4bfb96b700: Status 404 returned error can't find the container with id b5612e14998850453c591a731c3416a939a8194d0cce1ee7c692ee4bfb96b700
Jul 28 14:10:13.060783 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.060749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km5pf\" (UniqueName: \"kubernetes.io/projected/78cd0f77-4dde-42aa-960e-f88aa51127f6-kube-api-access-km5pf\") pod \"authorino-6ddb96c995-st476\" (UID: \"78cd0f77-4dde-42aa-960e-f88aa51127f6\") " pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:10:13.060915 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.060805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78cd0f77-4dde-42aa-960e-f88aa51127f6-tls-cert\") pod \"authorino-6ddb96c995-st476\" (UID: \"78cd0f77-4dde-42aa-960e-f88aa51127f6\") " pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:10:13.063333 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.063310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78cd0f77-4dde-42aa-960e-f88aa51127f6-tls-cert\") pod \"authorino-6ddb96c995-st476\" (UID: \"78cd0f77-4dde-42aa-960e-f88aa51127f6\") " pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:10:13.075245 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.075223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km5pf\" (UniqueName: \"kubernetes.io/projected/78cd0f77-4dde-42aa-960e-f88aa51127f6-kube-api-access-km5pf\") pod \"authorino-6ddb96c995-st476\" (UID: \"78cd0f77-4dde-42aa-960e-f88aa51127f6\") " pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:10:13.195298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.195221    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:10:13.347812 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.347762    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6ddb96c995-st476"]
Jul 28 14:10:13.350112 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:10:13.350084    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78cd0f77_4dde_42aa_960e_f88aa51127f6.slice/crio-af85b1e12ce1029af5da3fdd8291a77c6b4786ee8e9bc5c8041b015c262d0a18 WatchSource:0}: Error finding container af85b1e12ce1029af5da3fdd8291a77c6b4786ee8e9bc5c8041b015c262d0a18: Status 404 returned error can't find the container with id af85b1e12ce1029af5da3fdd8291a77c6b4786ee8e9bc5c8041b015c262d0a18
Jul 28 14:10:13.593174 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.593135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddb96c995-st476" event={"ID":"78cd0f77-4dde-42aa-960e-f88aa51127f6","Type":"ContainerStarted","Data":"af85b1e12ce1029af5da3fdd8291a77c6b4786ee8e9bc5c8041b015c262d0a18"}
Jul 28 14:10:13.594531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.594505    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7sq58" event={"ID":"9dae1b14-c8ff-4a17-a6d1-a4a7d592244a","Type":"ContainerStarted","Data":"9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac"}
Jul 28 14:10:13.594670 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.594536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7sq58" event={"ID":"9dae1b14-c8ff-4a17-a6d1-a4a7d592244a","Type":"ContainerStarted","Data":"97fe30d63d6a8588baf6075ff6251aab7cde733e31f4ff4e0e3b5f9d7f975675"}
Jul 28 14:10:13.594670 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.594549    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-7sq58" podUID="9dae1b14-c8ff-4a17-a6d1-a4a7d592244a" containerName="authorino" containerID="cri-o://9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac" gracePeriod=30
Jul 28 14:10:13.595890 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.595870    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-775959b5c9-g7qx5" event={"ID":"1b945827-7a5d-4b16-96dd-219fcd66c07c","Type":"ContainerStarted","Data":"b5612e14998850453c591a731c3416a939a8194d0cce1ee7c692ee4bfb96b700"}
Jul 28 14:10:13.625113 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.625071    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-7sq58" podStartSLOduration=1.037448362 podStartE2EDuration="1.625056959s" totalImagesPullingTime="587.608597ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:10:13.623972615 +0000 UTC m=+629.751363300" watchObservedRunningTime="2026-07-28 14:10:13.625056959 +0000 UTC m=+629.752447675"
Jul 28 14:10:13.851140 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.851088    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7sq58"
Jul 28 14:10:13.968220 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.968202    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dswqk\" (UniqueName: \"kubernetes.io/projected/9dae1b14-c8ff-4a17-a6d1-a4a7d592244a-kube-api-access-dswqk\") pod \"9dae1b14-c8ff-4a17-a6d1-a4a7d592244a\" (UID: \"9dae1b14-c8ff-4a17-a6d1-a4a7d592244a\") "
Jul 28 14:10:13.970379 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:13.970347    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dae1b14-c8ff-4a17-a6d1-a4a7d592244a-kube-api-access-dswqk" (OuterVolumeSpecName: "kube-api-access-dswqk") pod "9dae1b14-c8ff-4a17-a6d1-a4a7d592244a" (UID: "9dae1b14-c8ff-4a17-a6d1-a4a7d592244a"). InnerVolumeSpecName "kube-api-access-dswqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:10:14.069733 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.069691    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dswqk\" (UniqueName: \"kubernetes.io/projected/9dae1b14-c8ff-4a17-a6d1-a4a7d592244a-kube-api-access-dswqk\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:10:14.603229 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.603139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddb96c995-st476" event={"ID":"78cd0f77-4dde-42aa-960e-f88aa51127f6","Type":"ContainerStarted","Data":"88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81"}
Jul 28 14:10:14.604445 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.604410    2579 generic.go:356] "Generic (PLEG): container finished" podID="9dae1b14-c8ff-4a17-a6d1-a4a7d592244a" containerID="9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac" exitCode=0
Jul 28 14:10:14.604583 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.604464    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7sq58"
Jul 28 14:10:14.604583 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.604487    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7sq58" event={"ID":"9dae1b14-c8ff-4a17-a6d1-a4a7d592244a","Type":"ContainerDied","Data":"9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac"}
Jul 28 14:10:14.604583 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.604515    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7sq58" event={"ID":"9dae1b14-c8ff-4a17-a6d1-a4a7d592244a","Type":"ContainerDied","Data":"97fe30d63d6a8588baf6075ff6251aab7cde733e31f4ff4e0e3b5f9d7f975675"}
Jul 28 14:10:14.604583 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.604534    2579 scope.go:117] "RemoveContainer" containerID="9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac"
Jul 28 14:10:14.606062 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.606039    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-775959b5c9-g7qx5" event={"ID":"1b945827-7a5d-4b16-96dd-219fcd66c07c","Type":"ContainerStarted","Data":"eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0"}
Jul 28 14:10:14.606167 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.606086    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-775959b5c9-g7qx5" podUID="1b945827-7a5d-4b16-96dd-219fcd66c07c" containerName="authorino" containerID="cri-o://eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0" gracePeriod=30
Jul 28 14:10:14.614290 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.614268    2579 scope.go:117] "RemoveContainer" containerID="9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac"
Jul 28 14:10:14.614542 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:10:14.614525    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac\": container with ID starting with 9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac not found: ID does not exist" containerID="9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac"
Jul 28 14:10:14.614600 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.614550    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac"} err="failed to get container status \"9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac\": rpc error: code = NotFound desc = could not find container \"9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac\": container with ID starting with 9db9f5e3a5eb4b199cb0d9e6107804f968ec9e366a0d66a0d8e894862d9621ac not found: ID does not exist"
Jul 28 14:10:14.630641 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.630601    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6ddb96c995-st476" podStartSLOduration=1.670527624 podStartE2EDuration="2.630589245s" totalImagesPullingTime="960.061621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:10:14.627787786 +0000 UTC m=+630.755178468" watchObservedRunningTime="2026-07-28 14:10:14.630589245 +0000 UTC m=+630.757979936"
Jul 28 14:10:14.652884 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.652834    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-775959b5c9-g7qx5" podStartSLOduration=1.672892074 podStartE2EDuration="2.652815702s" totalImagesPullingTime="979.923628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:10:14.652076304 +0000 UTC m=+630.779466995" watchObservedRunningTime="2026-07-28 14:10:14.652815702 +0000 UTC m=+630.780206395"
Jul 28 14:10:14.670879 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.670854    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7sq58"]
Jul 28 14:10:14.681478 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.681451    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7sq58"]
Jul 28 14:10:14.863326 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.863271    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-775959b5c9-g7qx5"
Jul 28 14:10:14.979034 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.978986    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgfvn\" (UniqueName: \"kubernetes.io/projected/1b945827-7a5d-4b16-96dd-219fcd66c07c-kube-api-access-pgfvn\") pod \"1b945827-7a5d-4b16-96dd-219fcd66c07c\" (UID: \"1b945827-7a5d-4b16-96dd-219fcd66c07c\") "
Jul 28 14:10:14.981251 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:14.981223    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b945827-7a5d-4b16-96dd-219fcd66c07c-kube-api-access-pgfvn" (OuterVolumeSpecName: "kube-api-access-pgfvn") pod "1b945827-7a5d-4b16-96dd-219fcd66c07c" (UID: "1b945827-7a5d-4b16-96dd-219fcd66c07c"). InnerVolumeSpecName "kube-api-access-pgfvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:10:15.080102 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.080068    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgfvn\" (UniqueName: \"kubernetes.io/projected/1b945827-7a5d-4b16-96dd-219fcd66c07c-kube-api-access-pgfvn\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:10:15.613304 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.613272    2579 generic.go:356] "Generic (PLEG): container finished" podID="1b945827-7a5d-4b16-96dd-219fcd66c07c" containerID="eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0" exitCode=0
Jul 28 14:10:15.613711 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.613312    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-775959b5c9-g7qx5"
Jul 28 14:10:15.613711 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.613361    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-775959b5c9-g7qx5" event={"ID":"1b945827-7a5d-4b16-96dd-219fcd66c07c","Type":"ContainerDied","Data":"eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0"}
Jul 28 14:10:15.613711 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.613398    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-775959b5c9-g7qx5" event={"ID":"1b945827-7a5d-4b16-96dd-219fcd66c07c","Type":"ContainerDied","Data":"b5612e14998850453c591a731c3416a939a8194d0cce1ee7c692ee4bfb96b700"}
Jul 28 14:10:15.613711 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.613419    2579 scope.go:117] "RemoveContainer" containerID="eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0"
Jul 28 14:10:15.622617 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.622601    2579 scope.go:117] "RemoveContainer" containerID="eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0"
Jul 28 14:10:15.622880 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:10:15.622861    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0\": container with ID starting with eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0 not found: ID does not exist" containerID="eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0"
Jul 28 14:10:15.622927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.622887    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0"} err="failed to get container status \"eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0\": rpc error: code = NotFound desc = could not find container \"eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0\": container with ID starting with eccfb49b5181f5341091b86166cb1c71b51904707d92aba48eaedd0cac1a17f0 not found: ID does not exist"
Jul 28 14:10:15.645559 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.645537    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-775959b5c9-g7qx5"]
Jul 28 14:10:15.654770 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:15.654747    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-775959b5c9-g7qx5"]
Jul 28 14:10:16.342630 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:16.342592    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b945827-7a5d-4b16-96dd-219fcd66c07c" path="/var/lib/kubelet/pods/1b945827-7a5d-4b16-96dd-219fcd66c07c/volumes"
Jul 28 14:10:16.342928 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:16.342915    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dae1b14-c8ff-4a17-a6d1-a4a7d592244a" path="/var/lib/kubelet/pods/9dae1b14-c8ff-4a17-a6d1-a4a7d592244a/volumes"
Jul 28 14:10:57.725395 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:10:57.725363    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:11:15.541199 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:11:15.541168    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:11:23.844043 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:11:23.843983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:11:37.345283 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:11:37.345200    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:11:46.488171 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:11:46.488132    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:11:57.343663 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:11:57.343623    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:12:10.656612 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:12:10.656576    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:13:30.961935 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:30.961905    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8587bdf57d-gck8j"]
Jul 28 14:13:30.962374 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:30.962326    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dae1b14-c8ff-4a17-a6d1-a4a7d592244a" containerName="authorino"
Jul 28 14:13:30.962374 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:30.962339    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dae1b14-c8ff-4a17-a6d1-a4a7d592244a" containerName="authorino"
Jul 28 14:13:30.962374 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:30.962350    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b945827-7a5d-4b16-96dd-219fcd66c07c" containerName="authorino"
Jul 28 14:13:30.962374 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:30.962355    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b945827-7a5d-4b16-96dd-219fcd66c07c" containerName="authorino"
Jul 28 14:13:30.962504 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:30.962420    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b945827-7a5d-4b16-96dd-219fcd66c07c" containerName="authorino"
Jul 28 14:13:30.962504 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:30.962430    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dae1b14-c8ff-4a17-a6d1-a4a7d592244a" containerName="authorino"
Jul 28 14:13:30.965415 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:30.965400    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8587bdf57d-gck8j"
Jul 28 14:13:30.978974 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:30.978943    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8587bdf57d-gck8j"]
Jul 28 14:13:31.035794 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.035767    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4c61ce58-9932-4032-b998-6b8eacfd5834-tls-cert\") pod \"authorino-8587bdf57d-gck8j\" (UID: \"4c61ce58-9932-4032-b998-6b8eacfd5834\") " pod="kuadrant-system/authorino-8587bdf57d-gck8j"
Jul 28 14:13:31.035794 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.035797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrdzh\" (UniqueName: \"kubernetes.io/projected/4c61ce58-9932-4032-b998-6b8eacfd5834-kube-api-access-hrdzh\") pod \"authorino-8587bdf57d-gck8j\" (UID: \"4c61ce58-9932-4032-b998-6b8eacfd5834\") " pod="kuadrant-system/authorino-8587bdf57d-gck8j"
Jul 28 14:13:31.136322 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.136287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4c61ce58-9932-4032-b998-6b8eacfd5834-tls-cert\") pod \"authorino-8587bdf57d-gck8j\" (UID: \"4c61ce58-9932-4032-b998-6b8eacfd5834\") " pod="kuadrant-system/authorino-8587bdf57d-gck8j"
Jul 28 14:13:31.136512 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.136326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrdzh\" (UniqueName: \"kubernetes.io/projected/4c61ce58-9932-4032-b998-6b8eacfd5834-kube-api-access-hrdzh\") pod \"authorino-8587bdf57d-gck8j\" (UID: \"4c61ce58-9932-4032-b998-6b8eacfd5834\") " pod="kuadrant-system/authorino-8587bdf57d-gck8j"
Jul 28 14:13:31.138963 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.138930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4c61ce58-9932-4032-b998-6b8eacfd5834-tls-cert\") pod \"authorino-8587bdf57d-gck8j\" (UID: \"4c61ce58-9932-4032-b998-6b8eacfd5834\") " pod="kuadrant-system/authorino-8587bdf57d-gck8j"
Jul 28 14:13:31.147976 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.147948    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrdzh\" (UniqueName: \"kubernetes.io/projected/4c61ce58-9932-4032-b998-6b8eacfd5834-kube-api-access-hrdzh\") pod \"authorino-8587bdf57d-gck8j\" (UID: \"4c61ce58-9932-4032-b998-6b8eacfd5834\") " pod="kuadrant-system/authorino-8587bdf57d-gck8j"
Jul 28 14:13:31.274642 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.274606    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8587bdf57d-gck8j"
Jul 28 14:13:31.415885 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.415856    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8587bdf57d-gck8j"]
Jul 28 14:13:31.416649 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:13:31.416623    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c61ce58_9932_4032_b998_6b8eacfd5834.slice/crio-266250cc497545a8585d048e10bed435a2e4e4276151a6ad986987c3187279c6 WatchSource:0}: Error finding container 266250cc497545a8585d048e10bed435a2e4e4276151a6ad986987c3187279c6: Status 404 returned error can't find the container with id 266250cc497545a8585d048e10bed435a2e4e4276151a6ad986987c3187279c6
Jul 28 14:13:31.417943 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.417926    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:13:31.673871 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:31.673785    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8587bdf57d-gck8j" event={"ID":"4c61ce58-9932-4032-b998-6b8eacfd5834","Type":"ContainerStarted","Data":"266250cc497545a8585d048e10bed435a2e4e4276151a6ad986987c3187279c6"}
Jul 28 14:13:32.682483 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:32.682452    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8587bdf57d-gck8j" event={"ID":"4c61ce58-9932-4032-b998-6b8eacfd5834","Type":"ContainerStarted","Data":"453c3a3ee61e077bf1d2f5d0ea5a069ee376aaa37164c2a379d12c8b36f08910"}
Jul 28 14:13:32.719855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:32.719794    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8587bdf57d-gck8j" podStartSLOduration=2.194568341 podStartE2EDuration="2.719757235s" totalImagesPullingTime="525.188894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:13:32.713298503 +0000 UTC m=+828.840689196" watchObservedRunningTime="2026-07-28 14:13:32.719757235 +0000 UTC m=+828.847147928"
Jul 28 14:13:32.777417 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:32.777377    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6ddb96c995-st476"]
Jul 28 14:13:32.778090 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:32.777837    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6ddb96c995-st476" podUID="78cd0f77-4dde-42aa-960e-f88aa51127f6" containerName="authorino" containerID="cri-o://88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81" gracePeriod=30
Jul 28 14:13:33.035911 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.035880    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:13:33.156693 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.156658    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km5pf\" (UniqueName: \"kubernetes.io/projected/78cd0f77-4dde-42aa-960e-f88aa51127f6-kube-api-access-km5pf\") pod \"78cd0f77-4dde-42aa-960e-f88aa51127f6\" (UID: \"78cd0f77-4dde-42aa-960e-f88aa51127f6\") "
Jul 28 14:13:33.156858 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.156713    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78cd0f77-4dde-42aa-960e-f88aa51127f6-tls-cert\") pod \"78cd0f77-4dde-42aa-960e-f88aa51127f6\" (UID: \"78cd0f77-4dde-42aa-960e-f88aa51127f6\") "
Jul 28 14:13:33.158919 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.158883    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cd0f77-4dde-42aa-960e-f88aa51127f6-kube-api-access-km5pf" (OuterVolumeSpecName: "kube-api-access-km5pf") pod "78cd0f77-4dde-42aa-960e-f88aa51127f6" (UID: "78cd0f77-4dde-42aa-960e-f88aa51127f6"). InnerVolumeSpecName "kube-api-access-km5pf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:13:33.167710 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.167681    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78cd0f77-4dde-42aa-960e-f88aa51127f6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "78cd0f77-4dde-42aa-960e-f88aa51127f6" (UID: "78cd0f77-4dde-42aa-960e-f88aa51127f6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:13:33.258575 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.258508    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78cd0f77-4dde-42aa-960e-f88aa51127f6-tls-cert\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:13:33.258575 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.258546    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km5pf\" (UniqueName: \"kubernetes.io/projected/78cd0f77-4dde-42aa-960e-f88aa51127f6-kube-api-access-km5pf\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:13:33.688951 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.688866    2579 generic.go:356] "Generic (PLEG): container finished" podID="78cd0f77-4dde-42aa-960e-f88aa51127f6" containerID="88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81" exitCode=0
Jul 28 14:13:33.688951 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.688914    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddb96c995-st476"
Jul 28 14:13:33.689470 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.688955    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddb96c995-st476" event={"ID":"78cd0f77-4dde-42aa-960e-f88aa51127f6","Type":"ContainerDied","Data":"88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81"}
Jul 28 14:13:33.689470 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.689002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddb96c995-st476" event={"ID":"78cd0f77-4dde-42aa-960e-f88aa51127f6","Type":"ContainerDied","Data":"af85b1e12ce1029af5da3fdd8291a77c6b4786ee8e9bc5c8041b015c262d0a18"}
Jul 28 14:13:33.689470 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.689044    2579 scope.go:117] "RemoveContainer" containerID="88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81"
Jul 28 14:13:33.698757 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.698740    2579 scope.go:117] "RemoveContainer" containerID="88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81"
Jul 28 14:13:33.699105 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:13:33.699087    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81\": container with ID starting with 88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81 not found: ID does not exist" containerID="88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81"
Jul 28 14:13:33.699175 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.699118    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81"} err="failed to get container status \"88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81\": rpc error: code = NotFound desc = could not find container \"88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81\": container with ID starting with 88e6eb6740ac0d52316f6e174657452f41cdd41c499e711189ac9ce91fa49b81 not found: ID does not exist"
Jul 28 14:13:33.716407 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.716374    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6ddb96c995-st476"]
Jul 28 14:13:33.722685 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:33.722655    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6ddb96c995-st476"]
Jul 28 14:13:34.342193 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:13:34.342154    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78cd0f77-4dde-42aa-960e-f88aa51127f6" path="/var/lib/kubelet/pods/78cd0f77-4dde-42aa-960e-f88aa51127f6/volumes"
Jul 28 14:14:15.439796 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:14:15.439758    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:14:27.442305 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:14:27.442263    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:14:36.543560 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:14:36.543479    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:14:44.272168 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:14:44.272141    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:14:44.274519 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:14:44.274500    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:16:19.747277 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:16:19.747193    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:16:29.040169 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:16:29.040128    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:17:02.839307 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:17:02.839273    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:17:12.239715 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:17:12.239677    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:17:22.640966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:17:22.640932    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:17:28.536722 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:17:28.536679    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:17:38.839877 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:17:38.839795    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:17:49.053793 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:17:49.053756    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:18:01.440444 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:18:01.440413    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:18:11.736833 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:18:11.736798    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:18:33.053183 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:18:33.053148    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:18:44.843941 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:18:44.843905    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:19:09.447415 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:19:09.447322    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:19:21.743847 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:19:21.743810    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:19:32.052385 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:19:32.052350    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:19:44.309808 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:19:44.309781    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:19:44.312726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:19:44.312707    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:19:48.538679 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:19:48.538642    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:19:57.853675 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:19:57.853637    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:20:07.343208 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:20:07.343173    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:20:17.081712 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:20:17.081680    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:20:26.253609 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:20:26.253574    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:22:24.842190 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:22:24.842158    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:22:35.853215 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:22:35.853174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:22:51.844361 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:22:51.844319    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:23:03.663954 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:23:03.663916    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:23:14.447494 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:23:14.447464    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:23:25.044927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:23:25.044891    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:23:35.342168 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:23:35.342086    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:23:44.944093 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:23:44.944048    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:23:56.641004 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:23:56.640964    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:24:05.755243 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:24:05.755207    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:24:16.550232 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:24:16.550198    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:24:38.294247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:24:38.294212    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:24:44.351207 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:24:44.351166    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:24:44.356670 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:24:44.356645    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:24:48.150785 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:24:48.150753    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:24:58.452564 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:24:58.452530    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:25:14.669079 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:25:14.668939    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:25:34.786778 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:25:34.786744    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:25:50.354237 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:25:50.354203    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:25:57.456349 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:25:57.456319    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:26:14.065128 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:26:14.065095    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:26:28.685486 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:26:28.685450    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:26:38.372342 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:26:38.372264    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:26:51.850862 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:26:51.850824    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:27:02.555320 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:27:02.555284    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:27:15.750264 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:27:15.750225    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:27:25.544870 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:27:25.544840    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:27:35.442616 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:27:35.442580    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:27:47.844444 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:27:47.844407    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:27:57.939100 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:27:57.939063    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:28:08.144078 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:28:08.143980    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:28:19.156214 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:28:19.156181    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:28:31.547681 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:28:31.547642    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:28:45.245383 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:28:45.245349    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:28:59.148133 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:28:59.148060    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:29:11.747719 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:11.747682    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:29:29.168541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:29.168511    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:29:44.393271 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:44.393242    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:29:44.398389 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:44.398368    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:29:57.151648 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:57.151614    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:29:58.209199 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.209160    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"]
Jul 28 14:29:58.209594 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.209547    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78cd0f77-4dde-42aa-960e-f88aa51127f6" containerName="authorino"
Jul 28 14:29:58.209594 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.209557    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="78cd0f77-4dde-42aa-960e-f88aa51127f6" containerName="authorino"
Jul 28 14:29:58.209691 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.209619    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="78cd0f77-4dde-42aa-960e-f88aa51127f6" containerName="authorino"
Jul 28 14:29:58.212787 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.212772    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:29:58.215882 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.215857    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-d2089a60\""
Jul 28 14:29:58.231863 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.231839    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"]
Jul 28 14:29:58.295324 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.295296    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgwk8\" (UniqueName: \"kubernetes.io/projected/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-kube-api-access-sgwk8\") pod \"payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w\" (UID: \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\") " pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:29:58.295482 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.295335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w\" (UID: \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\") " pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:29:58.395966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.395936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwk8\" (UniqueName: \"kubernetes.io/projected/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-kube-api-access-sgwk8\") pod \"payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w\" (UID: \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\") " pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:29:58.396131 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.395980    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w\" (UID: \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\") " pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:29:58.396752 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.396729    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w\" (UID: \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\") " pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:29:58.408938 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.408918    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwk8\" (UniqueName: \"kubernetes.io/projected/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-kube-api-access-sgwk8\") pod \"payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w\" (UID: \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\") " pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:29:58.522817 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.522791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:29:58.678275 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.678246    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"]
Jul 28 14:29:58.679275 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:29:58.679248    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fbc36a_03a0_4292_bc51_ea8ada2fd396.slice/crio-553e45e74ce6c2fc945be3f9a3fac427e828687ce1cf9d122a8bb48d25ff3b8f WatchSource:0}: Error finding container 553e45e74ce6c2fc945be3f9a3fac427e828687ce1cf9d122a8bb48d25ff3b8f: Status 404 returned error can't find the container with id 553e45e74ce6c2fc945be3f9a3fac427e828687ce1cf9d122a8bb48d25ff3b8f
Jul 28 14:29:58.681099 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:58.681083    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:29:59.040086 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:29:59.040051    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w" event={"ID":"d6fbc36a-03a0-4292-bc51-ea8ada2fd396","Type":"ContainerStarted","Data":"553e45e74ce6c2fc945be3f9a3fac427e828687ce1cf9d122a8bb48d25ff3b8f"}
Jul 28 14:30:02.058068 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:02.058032    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w" event={"ID":"d6fbc36a-03a0-4292-bc51-ea8ada2fd396","Type":"ContainerStarted","Data":"131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066"}
Jul 28 14:30:02.058491 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:02.058143    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:30:02.082661 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:02.082611    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w" podStartSLOduration=1.736079302 podStartE2EDuration="4.082597212s" totalImagesPullingTime="2.34651791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:30:02.078979471 +0000 UTC m=+1818.206370163" watchObservedRunningTime="2026-07-28 14:30:02.082597212 +0000 UTC m=+1818.209987932"
Jul 28 14:30:09.048724 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:09.048686    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:30:13.067044 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:13.066984    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:30:17.706309 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:17.706274    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"]
Jul 28 14:30:17.706743 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:17.706492    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w" podUID="d6fbc36a-03a0-4292-bc51-ea8ada2fd396" containerName="payload-processing" containerID="cri-o://131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066" gracePeriod=30
Jul 28 14:30:17.964817 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:17.964752    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:30:17.968923 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:17.968899    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-plugins-config-volume\") pod \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\" (UID: \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\") "
Jul 28 14:30:17.969069 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:17.968993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgwk8\" (UniqueName: \"kubernetes.io/projected/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-kube-api-access-sgwk8\") pod \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\" (UID: \"d6fbc36a-03a0-4292-bc51-ea8ada2fd396\") "
Jul 28 14:30:17.969263 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:17.969240    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d6fbc36a-03a0-4292-bc51-ea8ada2fd396" (UID: "d6fbc36a-03a0-4292-bc51-ea8ada2fd396"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:30:17.971168 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:17.971146    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-kube-api-access-sgwk8" (OuterVolumeSpecName: "kube-api-access-sgwk8") pod "d6fbc36a-03a0-4292-bc51-ea8ada2fd396" (UID: "d6fbc36a-03a0-4292-bc51-ea8ada2fd396"). InnerVolumeSpecName "kube-api-access-sgwk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:30:18.069652 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.069617    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-plugins-config-volume\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:30:18.069652 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.069650    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgwk8\" (UniqueName: \"kubernetes.io/projected/d6fbc36a-03a0-4292-bc51-ea8ada2fd396-kube-api-access-sgwk8\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:30:18.152935 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.152897    2579 generic.go:356] "Generic (PLEG): container finished" podID="d6fbc36a-03a0-4292-bc51-ea8ada2fd396" containerID="131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066" exitCode=0
Jul 28 14:30:18.153139 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.152950    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w" event={"ID":"d6fbc36a-03a0-4292-bc51-ea8ada2fd396","Type":"ContainerDied","Data":"131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066"}
Jul 28 14:30:18.153139 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.152965    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"
Jul 28 14:30:18.153139 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.153005    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w" event={"ID":"d6fbc36a-03a0-4292-bc51-ea8ada2fd396","Type":"ContainerDied","Data":"553e45e74ce6c2fc945be3f9a3fac427e828687ce1cf9d122a8bb48d25ff3b8f"}
Jul 28 14:30:18.153139 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.153068    2579 scope.go:117] "RemoveContainer" containerID="131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066"
Jul 28 14:30:18.164475 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.164454    2579 scope.go:117] "RemoveContainer" containerID="131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066"
Jul 28 14:30:18.164772 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:30:18.164756    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066\": container with ID starting with 131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066 not found: ID does not exist" containerID="131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066"
Jul 28 14:30:18.164854 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.164778    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066"} err="failed to get container status \"131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066\": rpc error: code = NotFound desc = could not find container \"131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066\": container with ID starting with 131b6365949b0cf8a127a8802f1b47ae3be724b9bb5041e5f9fc7152dcb25066 not found: ID does not exist"
Jul 28 14:30:18.184796 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.184761    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"]
Jul 28 14:30:18.191843 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.191817    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d2089a60-8d5789cb9-5c78w"]
Jul 28 14:30:18.342233 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:18.342197    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6fbc36a-03a0-4292-bc51-ea8ada2fd396" path="/var/lib/kubelet/pods/d6fbc36a-03a0-4292-bc51-ea8ada2fd396/volumes"
Jul 28 14:30:22.070170 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.070133    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:30:22.681253 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.681214    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"]
Jul 28 14:30:22.681648 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.681632    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6fbc36a-03a0-4292-bc51-ea8ada2fd396" containerName="payload-processing"
Jul 28 14:30:22.681694 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.681650    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6fbc36a-03a0-4292-bc51-ea8ada2fd396" containerName="payload-processing"
Jul 28 14:30:22.681729 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.681721    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6fbc36a-03a0-4292-bc51-ea8ada2fd396" containerName="payload-processing"
Jul 28 14:30:22.686236 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.686216    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:30:22.688879 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.688858    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0b8a5700\""
Jul 28 14:30:22.688971 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.688886    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0b8a5700-dockercfg-skkxc\""
Jul 28 14:30:22.698129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.698103    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"]
Jul 28 14:30:22.713308 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.713280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86nww\" (UniqueName: \"kubernetes.io/projected/bc851832-1e91-47be-8ad5-06e84004c594-kube-api-access-86nww\") pod \"payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz\" (UID: \"bc851832-1e91-47be-8ad5-06e84004c594\") " pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:30:22.713473 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.713339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc851832-1e91-47be-8ad5-06e84004c594-plugins-config-volume\") pod \"payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz\" (UID: \"bc851832-1e91-47be-8ad5-06e84004c594\") " pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:30:22.814198 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.814165    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86nww\" (UniqueName: \"kubernetes.io/projected/bc851832-1e91-47be-8ad5-06e84004c594-kube-api-access-86nww\") pod \"payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz\" (UID: \"bc851832-1e91-47be-8ad5-06e84004c594\") " pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:30:22.814369 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.814207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc851832-1e91-47be-8ad5-06e84004c594-plugins-config-volume\") pod \"payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz\" (UID: \"bc851832-1e91-47be-8ad5-06e84004c594\") " pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:30:22.814838 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.814820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc851832-1e91-47be-8ad5-06e84004c594-plugins-config-volume\") pod \"payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz\" (UID: \"bc851832-1e91-47be-8ad5-06e84004c594\") " pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:30:22.824896 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.824868    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86nww\" (UniqueName: \"kubernetes.io/projected/bc851832-1e91-47be-8ad5-06e84004c594-kube-api-access-86nww\") pod \"payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz\" (UID: \"bc851832-1e91-47be-8ad5-06e84004c594\") " pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:30:22.998142 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:22.998050    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:30:23.151907 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:23.151876    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"]
Jul 28 14:30:23.154586 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:30:23.154561    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc851832_1e91_47be_8ad5_06e84004c594.slice/crio-9b41b26c5480bd0ecde652484547963c67294e8e621cf61adb8ba844797ca160 WatchSource:0}: Error finding container 9b41b26c5480bd0ecde652484547963c67294e8e621cf61adb8ba844797ca160: Status 404 returned error can't find the container with id 9b41b26c5480bd0ecde652484547963c67294e8e621cf61adb8ba844797ca160
Jul 28 14:30:23.182597 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:23.182573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz" event={"ID":"bc851832-1e91-47be-8ad5-06e84004c594","Type":"ContainerStarted","Data":"9b41b26c5480bd0ecde652484547963c67294e8e621cf61adb8ba844797ca160"}
Jul 28 14:30:24.190749 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:24.190716    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz" event={"ID":"bc851832-1e91-47be-8ad5-06e84004c594","Type":"ContainerStarted","Data":"ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1"}
Jul 28 14:30:24.191186 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:24.190797    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:30:24.216215 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:24.216155    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz" podStartSLOduration=2.216135054 podStartE2EDuration="2.216135054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:30:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:30:24.211551327 +0000 UTC m=+1840.338942017" watchObservedRunningTime="2026-07-28 14:30:24.216135054 +0000 UTC m=+1840.343525746"
Jul 28 14:30:33.351449 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:33.351408    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:30:35.197961 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:30:35.197933    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:31:03.590549 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:03.590513    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"]
Jul 28 14:31:03.591085 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:03.590716    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz" podUID="bc851832-1e91-47be-8ad5-06e84004c594" containerName="payload-processing" containerID="cri-o://ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1" gracePeriod=30
Jul 28 14:31:03.837524 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:03.837496    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:31:03.880359 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:03.880287    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc851832-1e91-47be-8ad5-06e84004c594-plugins-config-volume\") pod \"bc851832-1e91-47be-8ad5-06e84004c594\" (UID: \"bc851832-1e91-47be-8ad5-06e84004c594\") "
Jul 28 14:31:03.880359 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:03.880320    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86nww\" (UniqueName: \"kubernetes.io/projected/bc851832-1e91-47be-8ad5-06e84004c594-kube-api-access-86nww\") pod \"bc851832-1e91-47be-8ad5-06e84004c594\" (UID: \"bc851832-1e91-47be-8ad5-06e84004c594\") "
Jul 28 14:31:03.880711 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:03.880684    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc851832-1e91-47be-8ad5-06e84004c594-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc851832-1e91-47be-8ad5-06e84004c594" (UID: "bc851832-1e91-47be-8ad5-06e84004c594"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:31:03.882651 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:03.882622    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc851832-1e91-47be-8ad5-06e84004c594-kube-api-access-86nww" (OuterVolumeSpecName: "kube-api-access-86nww") pod "bc851832-1e91-47be-8ad5-06e84004c594" (UID: "bc851832-1e91-47be-8ad5-06e84004c594"). InnerVolumeSpecName "kube-api-access-86nww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:31:03.981084 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:03.981052    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc851832-1e91-47be-8ad5-06e84004c594-plugins-config-volume\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:31:03.981084 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:03.981083    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86nww\" (UniqueName: \"kubernetes.io/projected/bc851832-1e91-47be-8ad5-06e84004c594-kube-api-access-86nww\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:31:04.410206 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:04.410175    2579 generic.go:356] "Generic (PLEG): container finished" podID="bc851832-1e91-47be-8ad5-06e84004c594" containerID="ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1" exitCode=0
Jul 28 14:31:04.410387 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:04.410228    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"
Jul 28 14:31:04.410387 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:04.410260    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz" event={"ID":"bc851832-1e91-47be-8ad5-06e84004c594","Type":"ContainerDied","Data":"ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1"}
Jul 28 14:31:04.410387 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:04.410312    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz" event={"ID":"bc851832-1e91-47be-8ad5-06e84004c594","Type":"ContainerDied","Data":"9b41b26c5480bd0ecde652484547963c67294e8e621cf61adb8ba844797ca160"}
Jul 28 14:31:04.410387 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:04.410333    2579 scope.go:117] "RemoveContainer" containerID="ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1"
Jul 28 14:31:04.419511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:04.419493    2579 scope.go:117] "RemoveContainer" containerID="ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1"
Jul 28 14:31:04.419806 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:31:04.419788    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1\": container with ID starting with ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1 not found: ID does not exist" containerID="ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1"
Jul 28 14:31:04.419850 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:04.419817    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1"} err="failed to get container status \"ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1\": rpc error: code = NotFound desc = could not find container \"ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1\": container with ID starting with ab6246f7a3bb839a299964bb093b501d998fb8e820d93e0702483d06cf3dcde1 not found: ID does not exist"
Jul 28 14:31:04.437559 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:04.437536    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"]
Jul 28 14:31:04.446703 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:04.446677    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0b8a5700-7488866c8-drgvz"]
Jul 28 14:31:06.342789 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:06.342701    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc851832-1e91-47be-8ad5-06e84004c594" path="/var/lib/kubelet/pods/bc851832-1e91-47be-8ad5-06e84004c594/volumes"
Jul 28 14:31:07.656331 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:07.656286    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:31:10.700588 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.700559    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"]
Jul 28 14:31:10.700983 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.700954    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc851832-1e91-47be-8ad5-06e84004c594" containerName="payload-processing"
Jul 28 14:31:10.700983 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.700964    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc851832-1e91-47be-8ad5-06e84004c594" containerName="payload-processing"
Jul 28 14:31:10.701100 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.701054    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc851832-1e91-47be-8ad5-06e84004c594" containerName="payload-processing"
Jul 28 14:31:10.705701 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.705677    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:10.710625 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.710599    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-384b773a\""
Jul 28 14:31:10.710897 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.710877    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-384b773a-dockercfg-wc6fd\""
Jul 28 14:31:10.719119 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.719090    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"]
Jul 28 14:31:10.746446 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.746415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2816a321-dc4a-4946-af0a-ee274a0d7d45-plugins-config-volume\") pod \"payload-processing-e2e-derive-384b773a-8547f4df54-glmtc\" (UID: \"2816a321-dc4a-4946-af0a-ee274a0d7d45\") " pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:10.746600 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.746453    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vv9f\" (UniqueName: \"kubernetes.io/projected/2816a321-dc4a-4946-af0a-ee274a0d7d45-kube-api-access-6vv9f\") pod \"payload-processing-e2e-derive-384b773a-8547f4df54-glmtc\" (UID: \"2816a321-dc4a-4946-af0a-ee274a0d7d45\") " pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:10.847676 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.847639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2816a321-dc4a-4946-af0a-ee274a0d7d45-plugins-config-volume\") pod \"payload-processing-e2e-derive-384b773a-8547f4df54-glmtc\" (UID: \"2816a321-dc4a-4946-af0a-ee274a0d7d45\") " pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:10.847862 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.847688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vv9f\" (UniqueName: \"kubernetes.io/projected/2816a321-dc4a-4946-af0a-ee274a0d7d45-kube-api-access-6vv9f\") pod \"payload-processing-e2e-derive-384b773a-8547f4df54-glmtc\" (UID: \"2816a321-dc4a-4946-af0a-ee274a0d7d45\") " pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:10.848415 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.848397    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2816a321-dc4a-4946-af0a-ee274a0d7d45-plugins-config-volume\") pod \"payload-processing-e2e-derive-384b773a-8547f4df54-glmtc\" (UID: \"2816a321-dc4a-4946-af0a-ee274a0d7d45\") " pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:10.869996 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:10.869964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vv9f\" (UniqueName: \"kubernetes.io/projected/2816a321-dc4a-4946-af0a-ee274a0d7d45-kube-api-access-6vv9f\") pod \"payload-processing-e2e-derive-384b773a-8547f4df54-glmtc\" (UID: \"2816a321-dc4a-4946-af0a-ee274a0d7d45\") " pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:11.017257 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:11.017228    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:11.183868 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:11.183846    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"]
Jul 28 14:31:11.185317 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:31:11.185285    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2816a321_dc4a_4946_af0a_ee274a0d7d45.slice/crio-e75839b55586979996607f91e3233bf2ef9c4b9ab3d3691f5f8303a9ba9645c2 WatchSource:0}: Error finding container e75839b55586979996607f91e3233bf2ef9c4b9ab3d3691f5f8303a9ba9645c2: Status 404 returned error can't find the container with id e75839b55586979996607f91e3233bf2ef9c4b9ab3d3691f5f8303a9ba9645c2
Jul 28 14:31:11.451182 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:11.451086    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc" event={"ID":"2816a321-dc4a-4946-af0a-ee274a0d7d45","Type":"ContainerStarted","Data":"acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267"}
Jul 28 14:31:11.451182 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:11.451143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc" event={"ID":"2816a321-dc4a-4946-af0a-ee274a0d7d45","Type":"ContainerStarted","Data":"e75839b55586979996607f91e3233bf2ef9c4b9ab3d3691f5f8303a9ba9645c2"}
Jul 28 14:31:11.451378 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:11.451210    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:11.480995 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:11.480941    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc" podStartSLOduration=1.480925795 podStartE2EDuration="1.480925795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:31:11.480826474 +0000 UTC m=+1887.608217164" watchObservedRunningTime="2026-07-28 14:31:11.480925795 +0000 UTC m=+1887.608316485"
Jul 28 14:31:21.751463 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:21.751425    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:31:22.459248 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:22.459223    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:37.955531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:37.955498    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"]
Jul 28 14:31:37.956117 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:37.955778    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc" podUID="2816a321-dc4a-4946-af0a-ee274a0d7d45" containerName="payload-processing" containerID="cri-o://acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267" gracePeriod=30
Jul 28 14:31:38.208091 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.208010    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:38.390395 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.390360    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vv9f\" (UniqueName: \"kubernetes.io/projected/2816a321-dc4a-4946-af0a-ee274a0d7d45-kube-api-access-6vv9f\") pod \"2816a321-dc4a-4946-af0a-ee274a0d7d45\" (UID: \"2816a321-dc4a-4946-af0a-ee274a0d7d45\") "
Jul 28 14:31:38.390574 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.390447    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2816a321-dc4a-4946-af0a-ee274a0d7d45-plugins-config-volume\") pod \"2816a321-dc4a-4946-af0a-ee274a0d7d45\" (UID: \"2816a321-dc4a-4946-af0a-ee274a0d7d45\") "
Jul 28 14:31:38.390852 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.390824    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2816a321-dc4a-4946-af0a-ee274a0d7d45-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2816a321-dc4a-4946-af0a-ee274a0d7d45" (UID: "2816a321-dc4a-4946-af0a-ee274a0d7d45"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:31:38.392756 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.392737    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2816a321-dc4a-4946-af0a-ee274a0d7d45-kube-api-access-6vv9f" (OuterVolumeSpecName: "kube-api-access-6vv9f") pod "2816a321-dc4a-4946-af0a-ee274a0d7d45" (UID: "2816a321-dc4a-4946-af0a-ee274a0d7d45"). InnerVolumeSpecName "kube-api-access-6vv9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:31:38.492168 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.492083    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vv9f\" (UniqueName: \"kubernetes.io/projected/2816a321-dc4a-4946-af0a-ee274a0d7d45-kube-api-access-6vv9f\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:31:38.492168 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.492111    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2816a321-dc4a-4946-af0a-ee274a0d7d45-plugins-config-volume\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:31:38.598296 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.598265    2579 generic.go:356] "Generic (PLEG): container finished" podID="2816a321-dc4a-4946-af0a-ee274a0d7d45" containerID="acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267" exitCode=0
Jul 28 14:31:38.598473 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.598311    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"
Jul 28 14:31:38.598473 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.598358    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc" event={"ID":"2816a321-dc4a-4946-af0a-ee274a0d7d45","Type":"ContainerDied","Data":"acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267"}
Jul 28 14:31:38.598473 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.598413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc" event={"ID":"2816a321-dc4a-4946-af0a-ee274a0d7d45","Type":"ContainerDied","Data":"e75839b55586979996607f91e3233bf2ef9c4b9ab3d3691f5f8303a9ba9645c2"}
Jul 28 14:31:38.598473 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.598440    2579 scope.go:117] "RemoveContainer" containerID="acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267"
Jul 28 14:31:38.608191 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.608174    2579 scope.go:117] "RemoveContainer" containerID="acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267"
Jul 28 14:31:38.608484 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:31:38.608465    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267\": container with ID starting with acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267 not found: ID does not exist" containerID="acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267"
Jul 28 14:31:38.608525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.608493    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267"} err="failed to get container status \"acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267\": rpc error: code = NotFound desc = could not find container \"acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267\": container with ID starting with acdcbc8d6ec4368eee7615c4bf3a934f0d64274eba3e31da9f0a30c5b2e47267 not found: ID does not exist"
Jul 28 14:31:38.625849 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.625823    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"]
Jul 28 14:31:38.629703 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:38.629677    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-384b773a-8547f4df54-glmtc"]
Jul 28 14:31:40.342269 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:40.342234    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2816a321-dc4a-4946-af0a-ee274a0d7d45" path="/var/lib/kubelet/pods/2816a321-dc4a-4946-af0a-ee274a0d7d45/volumes"
Jul 28 14:31:42.050097 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:31:42.050065    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:33:09.566539 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:33:09.566500    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:33:22.184324 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:33:22.184294    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:33:31.456833 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:33:31.456791    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:33:42.843131 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:33:42.843095    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:34:27.239313 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:34:27.235870    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:34:44.430638 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:34:44.430603    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:34:44.436336 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:34:44.436312    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:35:01.036781 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:35:01.036745    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:35:40.041262 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:35:40.041172    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:36:00.746567 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:00.746535    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:36:05.251389 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.251354    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"]
Jul 28 14:36:05.251763 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.251736    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2816a321-dc4a-4946-af0a-ee274a0d7d45" containerName="payload-processing"
Jul 28 14:36:05.251763 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.251747    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2816a321-dc4a-4946-af0a-ee274a0d7d45" containerName="payload-processing"
Jul 28 14:36:05.251840 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.251808    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2816a321-dc4a-4946-af0a-ee274a0d7d45" containerName="payload-processing"
Jul 28 14:36:05.254983 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.254962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"
Jul 28 14:36:05.257250 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.257226    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 14:36:05.257356 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.257230    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:36:05.262844 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.262719    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"]
Jul 28 14:36:05.334692 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.334654    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369915d-b87f-4f8f-ae9b-0719a18744bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f2131889-gzsfl\" (UID: \"f369915d-b87f-4f8f-ae9b-0719a18744bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"
Jul 28 14:36:05.435754 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.435716    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369915d-b87f-4f8f-ae9b-0719a18744bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f2131889-gzsfl\" (UID: \"f369915d-b87f-4f8f-ae9b-0719a18744bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"
Jul 28 14:36:05.436389 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.436369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369915d-b87f-4f8f-ae9b-0719a18744bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f2131889-gzsfl\" (UID: \"f369915d-b87f-4f8f-ae9b-0719a18744bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"
Jul 28 14:36:05.566348 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.566316    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"
Jul 28 14:36:05.911752 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.911727    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"]
Jul 28 14:36:05.913331 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:36:05.913300    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf369915d_b87f_4f8f_ae9b_0719a18744bb.slice/crio-c3c332fcddd2b13c9465836b6785c7e6784374e4ae98e9516117b5037a8ef247 WatchSource:0}: Error finding container c3c332fcddd2b13c9465836b6785c7e6784374e4ae98e9516117b5037a8ef247: Status 404 returned error can't find the container with id c3c332fcddd2b13c9465836b6785c7e6784374e4ae98e9516117b5037a8ef247
Jul 28 14:36:05.915402 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:05.915383    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:36:06.035913 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:06.035870    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl" event={"ID":"f369915d-b87f-4f8f-ae9b-0719a18744bb","Type":"ContainerStarted","Data":"c3c332fcddd2b13c9465836b6785c7e6784374e4ae98e9516117b5037a8ef247"}
Jul 28 14:36:10.067285 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:10.067247    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl" event={"ID":"f369915d-b87f-4f8f-ae9b-0719a18744bb","Type":"ContainerStarted","Data":"7956a84f20bdb22745e0f11ed5ac9dd29febb215aa62fc1f605c6aa27b806dea"}
Jul 28 14:36:10.086779 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:10.086730    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl" podStartSLOduration=1.389330776 podStartE2EDuration="5.086713936s" totalImagesPullingTime="3.69738316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:36:10.084347136 +0000 UTC m=+2186.211737821" watchObservedRunningTime="2026-07-28 14:36:10.086713936 +0000 UTC m=+2186.214104626"
Jul 28 14:36:11.073805 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:11.073766    2579 generic.go:356] "Generic (PLEG): container finished" podID="f369915d-b87f-4f8f-ae9b-0719a18744bb" containerID="7956a84f20bdb22745e0f11ed5ac9dd29febb215aa62fc1f605c6aa27b806dea" exitCode=0
Jul 28 14:36:11.074295 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:11.073858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl" event={"ID":"f369915d-b87f-4f8f-ae9b-0719a18744bb","Type":"ContainerDied","Data":"7956a84f20bdb22745e0f11ed5ac9dd29febb215aa62fc1f605c6aa27b806dea"}
Jul 28 14:36:12.226924 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:12.226901    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"
Jul 28 14:36:12.291314 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:12.291284    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369915d-b87f-4f8f-ae9b-0719a18744bb-maas-api-service-ca\") pod \"f369915d-b87f-4f8f-ae9b-0719a18744bb\" (UID: \"f369915d-b87f-4f8f-ae9b-0719a18744bb\") "
Jul 28 14:36:12.291611 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:12.291589    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f369915d-b87f-4f8f-ae9b-0719a18744bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f369915d-b87f-4f8f-ae9b-0719a18744bb" (UID: "f369915d-b87f-4f8f-ae9b-0719a18744bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:36:12.392862 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:12.392833    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369915d-b87f-4f8f-ae9b-0719a18744bb-maas-api-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:36:13.087038 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:13.086988    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"
Jul 28 14:36:13.087038 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:13.087030    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl" event={"ID":"f369915d-b87f-4f8f-ae9b-0719a18744bb","Type":"ContainerDied","Data":"c3c332fcddd2b13c9465836b6785c7e6784374e4ae98e9516117b5037a8ef247"}
Jul 28 14:36:13.087241 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:13.087081    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3c332fcddd2b13c9465836b6785c7e6784374e4ae98e9516117b5037a8ef247"
Jul 28 14:36:19.539490 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:19.539453    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:36:21.026824 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:21.026787    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"]
Jul 28 14:36:21.029857 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:21.029834    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f2131889-gzsfl"]
Jul 28 14:36:22.341920 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:22.341881    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f369915d-b87f-4f8f-ae9b-0719a18744bb" path="/var/lib/kubelet/pods/f369915d-b87f-4f8f-ae9b-0719a18744bb/volumes"
Jul 28 14:36:30.331568 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:36:30.331534    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:37:41.965829 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:37:41.965745    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:37:51.843859 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:37:51.843824    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:38:44.429399 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:38:44.429327    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:39:07.800703 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.800669    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"]
Jul 28 14:39:07.801180 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.801076    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f369915d-b87f-4f8f-ae9b-0719a18744bb" containerName="revoke-keys"
Jul 28 14:39:07.801180 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.801089    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f369915d-b87f-4f8f-ae9b-0719a18744bb" containerName="revoke-keys"
Jul 28 14:39:07.801180 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.801180    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f369915d-b87f-4f8f-ae9b-0719a18744bb" containerName="revoke-keys"
Jul 28 14:39:07.804295 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.804276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:07.806692 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.806671    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-38f257-serving-cert\""
Jul 28 14:39:07.806790 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.806738    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 14:39:07.806790 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.806743    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hsfxf\""
Jul 28 14:39:07.807462 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.807434    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 14:39:07.817260 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.817242    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"]
Jul 28 14:39:07.869417 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.869386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr9dw\" (UniqueName: \"kubernetes.io/projected/a3c70d86-d1c4-4b96-90a7-08de529e1132-kube-api-access-kr9dw\") pod \"maas-api-e2e-inf-b-38f257-59674554c8-zj774\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:07.869615 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.869451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3c70d86-d1c4-4b96-90a7-08de529e1132-maas-api-tls\") pod \"maas-api-e2e-inf-b-38f257-59674554c8-zj774\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:07.869615 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.869523    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3c70d86-d1c4-4b96-90a7-08de529e1132-openshift-service-ca\") pod \"maas-api-e2e-inf-b-38f257-59674554c8-zj774\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:07.970371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.970340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3c70d86-d1c4-4b96-90a7-08de529e1132-openshift-service-ca\") pod \"maas-api-e2e-inf-b-38f257-59674554c8-zj774\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:07.970538 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.970435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr9dw\" (UniqueName: \"kubernetes.io/projected/a3c70d86-d1c4-4b96-90a7-08de529e1132-kube-api-access-kr9dw\") pod \"maas-api-e2e-inf-b-38f257-59674554c8-zj774\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:07.970538 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.970491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3c70d86-d1c4-4b96-90a7-08de529e1132-maas-api-tls\") pod \"maas-api-e2e-inf-b-38f257-59674554c8-zj774\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:07.971131 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.971111    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3c70d86-d1c4-4b96-90a7-08de529e1132-openshift-service-ca\") pod \"maas-api-e2e-inf-b-38f257-59674554c8-zj774\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:07.973048 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.973030    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3c70d86-d1c4-4b96-90a7-08de529e1132-maas-api-tls\") pod \"maas-api-e2e-inf-b-38f257-59674554c8-zj774\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:07.985474 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:07.985449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr9dw\" (UniqueName: \"kubernetes.io/projected/a3c70d86-d1c4-4b96-90a7-08de529e1132-kube-api-access-kr9dw\") pod \"maas-api-e2e-inf-b-38f257-59674554c8-zj774\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:08.115596 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:08.115500    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:08.255644 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:08.255617    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"]
Jul 28 14:39:08.257671 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:39:08.257645    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c70d86_d1c4_4b96_90a7_08de529e1132.slice/crio-63e9af55b93dcacd680bba4195950e73fb535e3b1df31c5ace940ab9604e04cb WatchSource:0}: Error finding container 63e9af55b93dcacd680bba4195950e73fb535e3b1df31c5ace940ab9604e04cb: Status 404 returned error can't find the container with id 63e9af55b93dcacd680bba4195950e73fb535e3b1df31c5ace940ab9604e04cb
Jul 28 14:39:09.040351 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:09.040311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774" event={"ID":"a3c70d86-d1c4-4b96-90a7-08de529e1132","Type":"ContainerStarted","Data":"63e9af55b93dcacd680bba4195950e73fb535e3b1df31c5ace940ab9604e04cb"}
Jul 28 14:39:11.054079 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:11.054045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774" event={"ID":"a3c70d86-d1c4-4b96-90a7-08de529e1132","Type":"ContainerStarted","Data":"da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114"}
Jul 28 14:39:11.054467 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:11.054156    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:11.072260 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:11.072207    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774" podStartSLOduration=1.6331932569999998 podStartE2EDuration="4.072192671s" totalImagesPullingTime="2.438999414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:39:11.071460802 +0000 UTC m=+2367.198851497" watchObservedRunningTime="2026-07-28 14:39:11.072192671 +0000 UTC m=+2367.199583362"
Jul 28 14:39:15.030089 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:15.030053    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:39:17.065129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:17.065101    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:39:39.133837 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.133801    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"]
Jul 28 14:39:39.137440 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.137420    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.140741 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.140713    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e8f53c\"/\"test-model-e8f53c-kserve-self-signed-certs\""
Jul 28 14:39:39.140860 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.140757    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e8f53c\"/\"default-dockercfg-jmgrw\""
Jul 28 14:39:39.140860 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.140854    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e8f53c\"/\"kube-root-ca.crt\""
Jul 28 14:39:39.140987 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.140777    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e8f53c\"/\"openshift-service-ca.crt\""
Jul 28 14:39:39.147827 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.147803    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"]
Jul 28 14:39:39.239669 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.239633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5vv5\" (UniqueName: \"kubernetes.io/projected/1da6adba-3717-4ed0-b604-656dad3378e5-kube-api-access-b5vv5\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.239845 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.239675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-tmp-dir\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.239845 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.239708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-home\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.239845 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.239806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-model-cache\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.239984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.239858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da6adba-3717-4ed0-b604-656dad3378e5-tls-certs\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.239984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.239913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-dshm\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.340729 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.340694    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da6adba-3717-4ed0-b604-656dad3378e5-tls-certs\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.340915 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.340736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-dshm\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.340915 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.340785    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5vv5\" (UniqueName: \"kubernetes.io/projected/1da6adba-3717-4ed0-b604-656dad3378e5-kube-api-access-b5vv5\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.340915 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.340811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-tmp-dir\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.340915 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.340841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-home\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.340915 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.340908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-model-cache\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.341379 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.341354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-model-cache\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.341379 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.341369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-tmp-dir\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.341731 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.341404    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-home\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.343285 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.343263    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-dshm\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.343417 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.343400    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da6adba-3717-4ed0-b604-656dad3378e5-tls-certs\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.349919 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.349898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5vv5\" (UniqueName: \"kubernetes.io/projected/1da6adba-3717-4ed0-b604-656dad3378e5-kube-api-access-b5vv5\") pod \"test-model-e8f53c-kserve-85db6885c-qxcml\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") " pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.448668 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.448595    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:39.583496 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:39.583471    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"]
Jul 28 14:39:39.587108 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:39:39.586201    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1da6adba_3717_4ed0_b604_656dad3378e5.slice/crio-d5fb7cc51ee9551e00db8e7667b96eac15df9cec782c17e6c362d6c29cf4d4c4 WatchSource:0}: Error finding container d5fb7cc51ee9551e00db8e7667b96eac15df9cec782c17e6c362d6c29cf4d4c4: Status 404 returned error can't find the container with id d5fb7cc51ee9551e00db8e7667b96eac15df9cec782c17e6c362d6c29cf4d4c4
Jul 28 14:39:40.213590 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:40.213552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml" event={"ID":"1da6adba-3717-4ed0-b604-656dad3378e5","Type":"ContainerStarted","Data":"d5fb7cc51ee9551e00db8e7667b96eac15df9cec782c17e6c362d6c29cf4d4c4"}
Jul 28 14:39:41.220609 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:41.220571    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml" event={"ID":"1da6adba-3717-4ed0-b604-656dad3378e5","Type":"ContainerStarted","Data":"193367e6b9eab9f0b2e7a4025bff8cfe315fe5a342618edd82deed100ac7c5d8"}
Jul 28 14:39:41.247667 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:41.247619    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml" podStartSLOduration=1.2535455309999999 podStartE2EDuration="2.247603045s" totalImagesPullingTime="994.057514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:39:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:39:41.245854128 +0000 UTC m=+2397.373244820" watchObservedRunningTime="2026-07-28 14:39:41.247603045 +0000 UTC m=+2397.374993736"
Jul 28 14:39:42.335437 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:42.335380    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:39:44.472512 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:44.472487    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:39:44.478436 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:44.478410    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:39:49.448802 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:49.448761    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:49.449247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:49.448818    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:49.460925 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:49.460902    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:50.280550 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:50.280523    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:39:54.031382 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:39:54.031347    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:40:21.942049 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:21.941946    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:40:36.041599 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:36.041569    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:40:57.521098 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:57.521065    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"]
Jul 28 14:40:58.707070 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:58.707031    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"]
Jul 28 14:40:58.710694 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:58.710673    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"
Jul 28 14:40:58.712797 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:58.712773    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:40:58.720091 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:58.720070    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"]
Jul 28 14:40:58.729986 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:58.729963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5417b07-88ec-4ed9-a87b-e871ecad54c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr\" (UID: \"d5417b07-88ec-4ed9-a87b-e871ecad54c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"
Jul 28 14:40:58.830602 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:58.830578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5417b07-88ec-4ed9-a87b-e871ecad54c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr\" (UID: \"d5417b07-88ec-4ed9-a87b-e871ecad54c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"
Jul 28 14:40:58.831159 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:58.831141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5417b07-88ec-4ed9-a87b-e871ecad54c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr\" (UID: \"d5417b07-88ec-4ed9-a87b-e871ecad54c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"
Jul 28 14:40:59.022740 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:59.022716    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"
Jul 28 14:40:59.157677 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:59.157654    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"]
Jul 28 14:40:59.160042 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:40:59.159985    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5417b07_88ec_4ed9_a87b_e871ecad54c3.slice/crio-b89eb9e69bc17186d2c523cd8eb98fb07f861634f81c1a0c8b3cabc155eb0576 WatchSource:0}: Error finding container b89eb9e69bc17186d2c523cd8eb98fb07f861634f81c1a0c8b3cabc155eb0576: Status 404 returned error can't find the container with id b89eb9e69bc17186d2c523cd8eb98fb07f861634f81c1a0c8b3cabc155eb0576
Jul 28 14:40:59.649739 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:59.649636    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr" event={"ID":"d5417b07-88ec-4ed9-a87b-e871ecad54c3","Type":"ContainerStarted","Data":"09f3123170d33d88ab93c5f88da6e64678c9ba361a6282d778d488c550780823"}
Jul 28 14:40:59.649739 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:59.649728    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr" event={"ID":"d5417b07-88ec-4ed9-a87b-e871ecad54c3","Type":"ContainerStarted","Data":"b89eb9e69bc17186d2c523cd8eb98fb07f861634f81c1a0c8b3cabc155eb0576"}
Jul 28 14:40:59.668447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:40:59.668397    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr" podStartSLOduration=1.668384293 podStartE2EDuration="1.668384293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:40:59.666945461 +0000 UTC m=+2475.794336147" watchObservedRunningTime="2026-07-28 14:40:59.668384293 +0000 UTC m=+2475.795775001"
Jul 28 14:41:00.657036 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:00.656987    2579 generic.go:356] "Generic (PLEG): container finished" podID="d5417b07-88ec-4ed9-a87b-e871ecad54c3" containerID="09f3123170d33d88ab93c5f88da6e64678c9ba361a6282d778d488c550780823" exitCode=0
Jul 28 14:41:00.657467 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:00.657077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr" event={"ID":"d5417b07-88ec-4ed9-a87b-e871ecad54c3","Type":"ContainerDied","Data":"09f3123170d33d88ab93c5f88da6e64678c9ba361a6282d778d488c550780823"}
Jul 28 14:41:01.799795 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:01.799766    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"
Jul 28 14:41:01.857438 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:01.857411    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5417b07-88ec-4ed9-a87b-e871ecad54c3-maas-api-service-ca\") pod \"d5417b07-88ec-4ed9-a87b-e871ecad54c3\" (UID: \"d5417b07-88ec-4ed9-a87b-e871ecad54c3\") "
Jul 28 14:41:01.857739 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:01.857715    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5417b07-88ec-4ed9-a87b-e871ecad54c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d5417b07-88ec-4ed9-a87b-e871ecad54c3" (UID: "d5417b07-88ec-4ed9-a87b-e871ecad54c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:41:01.958159 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:01.958078    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5417b07-88ec-4ed9-a87b-e871ecad54c3-maas-api-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:02.669971 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:02.669889    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"
Jul 28 14:41:02.669971 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:02.669899    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr" event={"ID":"d5417b07-88ec-4ed9-a87b-e871ecad54c3","Type":"ContainerDied","Data":"b89eb9e69bc17186d2c523cd8eb98fb07f861634f81c1a0c8b3cabc155eb0576"}
Jul 28 14:41:02.669971 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:02.669951    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b89eb9e69bc17186d2c523cd8eb98fb07f861634f81c1a0c8b3cabc155eb0576"
Jul 28 14:41:02.737662 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:02.737632    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:41:09.654729 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:09.654694    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"]
Jul 28 14:41:09.657877 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:09.657855    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e8f53c-dksxr"]
Jul 28 14:41:10.342651 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:10.342620    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5417b07-88ec-4ed9-a87b-e871ecad54c3" path="/var/lib/kubelet/pods/d5417b07-88ec-4ed9-a87b-e871ecad54c3/volumes"
Jul 28 14:41:12.540956 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.540914    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml" podUID="1da6adba-3717-4ed0-b604-656dad3378e5" containerName="main" containerID="cri-o://193367e6b9eab9f0b2e7a4025bff8cfe315fe5a342618edd82deed100ac7c5d8" gracePeriod=45
Jul 28 14:41:12.730457 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.730419    2579 generic.go:356] "Generic (PLEG): container finished" podID="1da6adba-3717-4ed0-b604-656dad3378e5" containerID="193367e6b9eab9f0b2e7a4025bff8cfe315fe5a342618edd82deed100ac7c5d8" exitCode=0
Jul 28 14:41:12.730679 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.730477    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml" event={"ID":"1da6adba-3717-4ed0-b604-656dad3378e5","Type":"ContainerDied","Data":"193367e6b9eab9f0b2e7a4025bff8cfe315fe5a342618edd82deed100ac7c5d8"}
Jul 28 14:41:12.795400 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.795341    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:41:12.861269 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861240    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da6adba-3717-4ed0-b604-656dad3378e5-tls-certs\") pod \"1da6adba-3717-4ed0-b604-656dad3378e5\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") "
Jul 28 14:41:12.861466 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861284    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-model-cache\") pod \"1da6adba-3717-4ed0-b604-656dad3378e5\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") "
Jul 28 14:41:12.861466 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861333    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-tmp-dir\") pod \"1da6adba-3717-4ed0-b604-656dad3378e5\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") "
Jul 28 14:41:12.861466 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861361    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-dshm\") pod \"1da6adba-3717-4ed0-b604-656dad3378e5\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") "
Jul 28 14:41:12.861466 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861394    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5vv5\" (UniqueName: \"kubernetes.io/projected/1da6adba-3717-4ed0-b604-656dad3378e5-kube-api-access-b5vv5\") pod \"1da6adba-3717-4ed0-b604-656dad3378e5\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") "
Jul 28 14:41:12.861466 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861441    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-home\") pod \"1da6adba-3717-4ed0-b604-656dad3378e5\" (UID: \"1da6adba-3717-4ed0-b604-656dad3378e5\") "
Jul 28 14:41:12.861720 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861636    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1da6adba-3717-4ed0-b604-656dad3378e5" (UID: "1da6adba-3717-4ed0-b604-656dad3378e5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:41:12.861720 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861643    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-model-cache" (OuterVolumeSpecName: "model-cache") pod "1da6adba-3717-4ed0-b604-656dad3378e5" (UID: "1da6adba-3717-4ed0-b604-656dad3378e5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:41:12.861842 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861793    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-home" (OuterVolumeSpecName: "home") pod "1da6adba-3717-4ed0-b604-656dad3378e5" (UID: "1da6adba-3717-4ed0-b604-656dad3378e5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:41:12.861899 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861868    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-model-cache\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:12.861899 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.861886    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-tmp-dir\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:12.863854 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.863828    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da6adba-3717-4ed0-b604-656dad3378e5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1da6adba-3717-4ed0-b604-656dad3378e5" (UID: "1da6adba-3717-4ed0-b604-656dad3378e5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:41:12.863854 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.863830    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-dshm" (OuterVolumeSpecName: "dshm") pod "1da6adba-3717-4ed0-b604-656dad3378e5" (UID: "1da6adba-3717-4ed0-b604-656dad3378e5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:41:12.864052 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.863828    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da6adba-3717-4ed0-b604-656dad3378e5-kube-api-access-b5vv5" (OuterVolumeSpecName: "kube-api-access-b5vv5") pod "1da6adba-3717-4ed0-b604-656dad3378e5" (UID: "1da6adba-3717-4ed0-b604-656dad3378e5"). InnerVolumeSpecName "kube-api-access-b5vv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:41:12.962888 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.962854    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-dshm\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:12.962888 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.962882    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5vv5\" (UniqueName: \"kubernetes.io/projected/1da6adba-3717-4ed0-b604-656dad3378e5-kube-api-access-b5vv5\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:12.962888 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.962896    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da6adba-3717-4ed0-b604-656dad3378e5-home\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:12.963157 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:12.962918    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da6adba-3717-4ed0-b604-656dad3378e5-tls-certs\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:13.737222 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:13.737193    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"
Jul 28 14:41:13.737602 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:13.737219    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml" event={"ID":"1da6adba-3717-4ed0-b604-656dad3378e5","Type":"ContainerDied","Data":"d5fb7cc51ee9551e00db8e7667b96eac15df9cec782c17e6c362d6c29cf4d4c4"}
Jul 28 14:41:13.737602 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:13.737272    2579 scope.go:117] "RemoveContainer" containerID="193367e6b9eab9f0b2e7a4025bff8cfe315fe5a342618edd82deed100ac7c5d8"
Jul 28 14:41:13.760441 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:13.760418    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"]
Jul 28 14:41:13.765248 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:13.765227    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e8f53c/test-model-e8f53c-kserve-85db6885c-qxcml"]
Jul 28 14:41:14.342117 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:14.342086    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1da6adba-3717-4ed0-b604-656dad3378e5" path="/var/lib/kubelet/pods/1da6adba-3717-4ed0-b604-656dad3378e5/volumes"
Jul 28 14:41:19.636611 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:19.636575    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:41:23.122782 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.122752    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"]
Jul 28 14:41:23.123198 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.122986    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774" podUID="a3c70d86-d1c4-4b96-90a7-08de529e1132" containerName="maas-api" containerID="cri-o://da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114" gracePeriod=30
Jul 28 14:41:23.375619 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.375556    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:41:23.455331 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.455296    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3c70d86-d1c4-4b96-90a7-08de529e1132-maas-api-tls\") pod \"a3c70d86-d1c4-4b96-90a7-08de529e1132\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") "
Jul 28 14:41:23.455498 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.455349    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3c70d86-d1c4-4b96-90a7-08de529e1132-openshift-service-ca\") pod \"a3c70d86-d1c4-4b96-90a7-08de529e1132\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") "
Jul 28 14:41:23.455498 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.455389    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr9dw\" (UniqueName: \"kubernetes.io/projected/a3c70d86-d1c4-4b96-90a7-08de529e1132-kube-api-access-kr9dw\") pod \"a3c70d86-d1c4-4b96-90a7-08de529e1132\" (UID: \"a3c70d86-d1c4-4b96-90a7-08de529e1132\") "
Jul 28 14:41:23.455964 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.455737    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3c70d86-d1c4-4b96-90a7-08de529e1132-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3c70d86-d1c4-4b96-90a7-08de529e1132" (UID: "a3c70d86-d1c4-4b96-90a7-08de529e1132"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:41:23.457831 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.457803    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3c70d86-d1c4-4b96-90a7-08de529e1132-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3c70d86-d1c4-4b96-90a7-08de529e1132" (UID: "a3c70d86-d1c4-4b96-90a7-08de529e1132"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:41:23.457935 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.457844    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3c70d86-d1c4-4b96-90a7-08de529e1132-kube-api-access-kr9dw" (OuterVolumeSpecName: "kube-api-access-kr9dw") pod "a3c70d86-d1c4-4b96-90a7-08de529e1132" (UID: "a3c70d86-d1c4-4b96-90a7-08de529e1132"). InnerVolumeSpecName "kube-api-access-kr9dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:41:23.557164 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.557131    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3c70d86-d1c4-4b96-90a7-08de529e1132-maas-api-tls\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:23.557164 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.557158    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3c70d86-d1c4-4b96-90a7-08de529e1132-openshift-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:23.557164 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.557169    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr9dw\" (UniqueName: \"kubernetes.io/projected/a3c70d86-d1c4-4b96-90a7-08de529e1132-kube-api-access-kr9dw\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:41:23.795673 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.795642    2579 generic.go:356] "Generic (PLEG): container finished" podID="a3c70d86-d1c4-4b96-90a7-08de529e1132" containerID="da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114" exitCode=0
Jul 28 14:41:23.795890 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.795687    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774" event={"ID":"a3c70d86-d1c4-4b96-90a7-08de529e1132","Type":"ContainerDied","Data":"da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114"}
Jul 28 14:41:23.795890 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.795696    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"
Jul 28 14:41:23.795890 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.795726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774" event={"ID":"a3c70d86-d1c4-4b96-90a7-08de529e1132","Type":"ContainerDied","Data":"63e9af55b93dcacd680bba4195950e73fb535e3b1df31c5ace940ab9604e04cb"}
Jul 28 14:41:23.795890 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.795749    2579 scope.go:117] "RemoveContainer" containerID="da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114"
Jul 28 14:41:23.805443 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.805427    2579 scope.go:117] "RemoveContainer" containerID="da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114"
Jul 28 14:41:23.805731 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:41:23.805712    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114\": container with ID starting with da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114 not found: ID does not exist" containerID="da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114"
Jul 28 14:41:23.805782 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.805741    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114"} err="failed to get container status \"da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114\": rpc error: code = NotFound desc = could not find container \"da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114\": container with ID starting with da0c717ad7d40353cc16b1a965116f58a7413f252ad0ade397eadf9648bba114 not found: ID does not exist"
Jul 28 14:41:23.818902 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.818878    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"]
Jul 28 14:41:23.826204 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:23.826183    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-38f257-59674554c8-zj774"]
Jul 28 14:41:24.348675 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:24.348644    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3c70d86-d1c4-4b96-90a7-08de529e1132" path="/var/lib/kubelet/pods/a3c70d86-d1c4-4b96-90a7-08de529e1132/volumes"
Jul 28 14:41:32.735268 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:32.735232    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:41:44.533901 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:41:44.533822    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:42:10.634938 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:10.634888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:42:31.798410 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.798374    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"]
Jul 28 14:42:31.798987 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.798948    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5417b07-88ec-4ed9-a87b-e871ecad54c3" containerName="revoke-keys"
Jul 28 14:42:31.798987 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.798967    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5417b07-88ec-4ed9-a87b-e871ecad54c3" containerName="revoke-keys"
Jul 28 14:42:31.798987 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.798978    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3c70d86-d1c4-4b96-90a7-08de529e1132" containerName="maas-api"
Jul 28 14:42:31.798987 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.798986    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3c70d86-d1c4-4b96-90a7-08de529e1132" containerName="maas-api"
Jul 28 14:42:31.799238 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.799032    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da6adba-3717-4ed0-b604-656dad3378e5" containerName="main"
Jul 28 14:42:31.799238 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.799041    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da6adba-3717-4ed0-b604-656dad3378e5" containerName="main"
Jul 28 14:42:31.799238 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.799153    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3c70d86-d1c4-4b96-90a7-08de529e1132" containerName="maas-api"
Jul 28 14:42:31.799238 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.799165    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da6adba-3717-4ed0-b604-656dad3378e5" containerName="main"
Jul 28 14:42:31.799238 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.799175    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5417b07-88ec-4ed9-a87b-e871ecad54c3" containerName="revoke-keys"
Jul 28 14:42:31.802520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.802491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"
Jul 28 14:42:31.805053 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.805011    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 14:42:31.805187 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.805143    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:42:31.810755 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.810531    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"]
Jul 28 14:42:31.867422 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.867389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5e49d3-9679-40c3-9d6d-8eb1da144ae7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk\" (UID: \"7f5e49d3-9679-40c3-9d6d-8eb1da144ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"
Jul 28 14:42:31.968786 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.968737    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5e49d3-9679-40c3-9d6d-8eb1da144ae7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk\" (UID: \"7f5e49d3-9679-40c3-9d6d-8eb1da144ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"
Jul 28 14:42:31.969416 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:31.969390    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5e49d3-9679-40c3-9d6d-8eb1da144ae7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk\" (UID: \"7f5e49d3-9679-40c3-9d6d-8eb1da144ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"
Jul 28 14:42:32.115902 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:32.115819    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"
Jul 28 14:42:32.278209 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:32.278181    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"]
Jul 28 14:42:32.279972 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:42:32.279949    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5e49d3_9679_40c3_9d6d_8eb1da144ae7.slice/crio-78b9655083929e8b4febcc1364a54dc06601cad459618fb53b34a1564a592cfe WatchSource:0}: Error finding container 78b9655083929e8b4febcc1364a54dc06601cad459618fb53b34a1564a592cfe: Status 404 returned error can't find the container with id 78b9655083929e8b4febcc1364a54dc06601cad459618fb53b34a1564a592cfe
Jul 28 14:42:33.174513 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:33.174480    2579 generic.go:356] "Generic (PLEG): container finished" podID="7f5e49d3-9679-40c3-9d6d-8eb1da144ae7" containerID="6cba5c0166e37a96e267917c79ce9b86398808b8a8cabefafc5e7c1d9009dd99" exitCode=0
Jul 28 14:42:33.174889 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:33.174557    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk" event={"ID":"7f5e49d3-9679-40c3-9d6d-8eb1da144ae7","Type":"ContainerDied","Data":"6cba5c0166e37a96e267917c79ce9b86398808b8a8cabefafc5e7c1d9009dd99"}
Jul 28 14:42:33.174889 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:33.174614    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk" event={"ID":"7f5e49d3-9679-40c3-9d6d-8eb1da144ae7","Type":"ContainerStarted","Data":"78b9655083929e8b4febcc1364a54dc06601cad459618fb53b34a1564a592cfe"}
Jul 28 14:42:34.316454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:34.316431    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"
Jul 28 14:42:34.390672 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:34.390645    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5e49d3-9679-40c3-9d6d-8eb1da144ae7-maas-api-service-ca\") pod \"7f5e49d3-9679-40c3-9d6d-8eb1da144ae7\" (UID: \"7f5e49d3-9679-40c3-9d6d-8eb1da144ae7\") "
Jul 28 14:42:34.391010 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:34.390991    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f5e49d3-9679-40c3-9d6d-8eb1da144ae7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f5e49d3-9679-40c3-9d6d-8eb1da144ae7" (UID: "7f5e49d3-9679-40c3-9d6d-8eb1da144ae7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:42:34.492157 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:34.492077    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5e49d3-9679-40c3-9d6d-8eb1da144ae7-maas-api-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:42:35.186330 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:35.186301    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"
Jul 28 14:42:35.186330 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:35.186325    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk" event={"ID":"7f5e49d3-9679-40c3-9d6d-8eb1da144ae7","Type":"ContainerDied","Data":"78b9655083929e8b4febcc1364a54dc06601cad459618fb53b34a1564a592cfe"}
Jul 28 14:42:35.186548 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:35.186365    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78b9655083929e8b4febcc1364a54dc06601cad459618fb53b34a1564a592cfe"
Jul 28 14:42:47.703369 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:47.703337    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"]
Jul 28 14:42:47.706307 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:47.706284    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4e9663-f6pkk"]
Jul 28 14:42:48.341400 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:48.341368    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f5e49d3-9679-40c3-9d6d-8eb1da144ae7" path="/var/lib/kubelet/pods/7f5e49d3-9679-40c3-9d6d-8eb1da144ae7/volumes"
Jul 28 14:42:50.030042 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:50.029996    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:42:52.430003 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:52.429972    2579 scope.go:117] "RemoveContainer" containerID="7956a84f20bdb22745e0f11ed5ac9dd29febb215aa62fc1f605c6aa27b806dea"
Jul 28 14:42:55.035442 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.035407    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"]
Jul 28 14:42:55.035821 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.035778    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f5e49d3-9679-40c3-9d6d-8eb1da144ae7" containerName="revoke-keys"
Jul 28 14:42:55.035821 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.035789    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f5e49d3-9679-40c3-9d6d-8eb1da144ae7" containerName="revoke-keys"
Jul 28 14:42:55.035893 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.035870    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f5e49d3-9679-40c3-9d6d-8eb1da144ae7" containerName="revoke-keys"
Jul 28 14:42:55.040560 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.040543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"
Jul 28 14:42:55.042744 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.042713    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 14:42:55.042744 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.042739    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:42:55.045910 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.045888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"]
Jul 28 14:42:55.170655 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.170620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d7214d-014f-4a5d-be42-eee46d6aa8ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb\" (UID: \"73d7214d-014f-4a5d-be42-eee46d6aa8ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"
Jul 28 14:42:55.271430 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.271394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d7214d-014f-4a5d-be42-eee46d6aa8ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb\" (UID: \"73d7214d-014f-4a5d-be42-eee46d6aa8ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"
Jul 28 14:42:55.272058 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.272038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d7214d-014f-4a5d-be42-eee46d6aa8ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb\" (UID: \"73d7214d-014f-4a5d-be42-eee46d6aa8ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"
Jul 28 14:42:55.352217 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.352131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"
Jul 28 14:42:55.484526 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:55.484480    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"]
Jul 28 14:42:55.486830 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:42:55.486802    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d7214d_014f_4a5d_be42_eee46d6aa8ad.slice/crio-4627c547cbfdd52be50a7e0255ccefe7d28738f77d44d333753e2a8ca54c85bc WatchSource:0}: Error finding container 4627c547cbfdd52be50a7e0255ccefe7d28738f77d44d333753e2a8ca54c85bc: Status 404 returned error can't find the container with id 4627c547cbfdd52be50a7e0255ccefe7d28738f77d44d333753e2a8ca54c85bc
Jul 28 14:42:56.305733 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:56.305700    2579 generic.go:356] "Generic (PLEG): container finished" podID="73d7214d-014f-4a5d-be42-eee46d6aa8ad" containerID="2fbd6b41e33d0a4c44384c8f957cb60f8a029743f156892d6c3b8838ad4ae253" exitCode=0
Jul 28 14:42:56.306129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:56.305751    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb" event={"ID":"73d7214d-014f-4a5d-be42-eee46d6aa8ad","Type":"ContainerDied","Data":"2fbd6b41e33d0a4c44384c8f957cb60f8a029743f156892d6c3b8838ad4ae253"}
Jul 28 14:42:56.306129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:56.305790    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb" event={"ID":"73d7214d-014f-4a5d-be42-eee46d6aa8ad","Type":"ContainerStarted","Data":"4627c547cbfdd52be50a7e0255ccefe7d28738f77d44d333753e2a8ca54c85bc"}
Jul 28 14:42:57.456688 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:57.456662    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"
Jul 28 14:42:57.592444 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:57.592360    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d7214d-014f-4a5d-be42-eee46d6aa8ad-maas-api-service-ca\") pod \"73d7214d-014f-4a5d-be42-eee46d6aa8ad\" (UID: \"73d7214d-014f-4a5d-be42-eee46d6aa8ad\") "
Jul 28 14:42:57.592701 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:57.592677    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73d7214d-014f-4a5d-be42-eee46d6aa8ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73d7214d-014f-4a5d-be42-eee46d6aa8ad" (UID: "73d7214d-014f-4a5d-be42-eee46d6aa8ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:42:57.693414 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:57.693382    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d7214d-014f-4a5d-be42-eee46d6aa8ad-maas-api-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:42:58.318499 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:58.318455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb" event={"ID":"73d7214d-014f-4a5d-be42-eee46d6aa8ad","Type":"ContainerDied","Data":"4627c547cbfdd52be50a7e0255ccefe7d28738f77d44d333753e2a8ca54c85bc"}
Jul 28 14:42:58.318696 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:58.318541    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4627c547cbfdd52be50a7e0255ccefe7d28738f77d44d333753e2a8ca54c85bc"
Jul 28 14:42:58.318696 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:42:58.318472    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"
Jul 28 14:43:10.928432 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:43:10.928355    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"]
Jul 28 14:43:10.931536 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:43:10.931513    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a409a3-sgjfb"]
Jul 28 14:43:12.341573 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:43:12.341537    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73d7214d-014f-4a5d-be42-eee46d6aa8ad" path="/var/lib/kubelet/pods/73d7214d-014f-4a5d-be42-eee46d6aa8ad/volumes"
Jul 28 14:43:15.737414 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:43:15.737377    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:43:26.038448 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:43:26.038402    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:44:04.256146 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:04.256102    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:44:25.519647 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.519614    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"]
Jul 28 14:44:25.520117 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.520045    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73d7214d-014f-4a5d-be42-eee46d6aa8ad" containerName="revoke-keys"
Jul 28 14:44:25.520117 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.520058    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="73d7214d-014f-4a5d-be42-eee46d6aa8ad" containerName="revoke-keys"
Jul 28 14:44:25.520193 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.520135    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="73d7214d-014f-4a5d-be42-eee46d6aa8ad" containerName="revoke-keys"
Jul 28 14:44:25.523093 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.523076    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.526240 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.526217    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-910fb6\"/\"kube-root-ca.crt\""
Jul 28 14:44:25.526240 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.526238    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-910fb6\"/\"auth-test-model-910fb6-kserve-self-signed-certs\""
Jul 28 14:44:25.526428 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.526263    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-910fb6\"/\"openshift-service-ca.crt\""
Jul 28 14:44:25.526428 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.526220    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-910fb6\"/\"default-dockercfg-hnx2r\""
Jul 28 14:44:25.534805 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.534781    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"]
Jul 28 14:44:25.582678 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.582642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-model-cache\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.582678 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.582683    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-home\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.582901 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.582779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-tmp-dir\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.582901 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.582830    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29d5344-6b64-4d26-9b3c-a51098a47812-tls-certs\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.582901 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.582851    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-dshm\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.582901 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.582875    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jscdl\" (UniqueName: \"kubernetes.io/projected/e29d5344-6b64-4d26-9b3c-a51098a47812-kube-api-access-jscdl\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.684298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.684262    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-model-cache\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.684298 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.684302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-home\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.684539 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.684351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-tmp-dir\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.684539 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.684374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29d5344-6b64-4d26-9b3c-a51098a47812-tls-certs\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.684539 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.684394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-dshm\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.684539 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.684425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jscdl\" (UniqueName: \"kubernetes.io/projected/e29d5344-6b64-4d26-9b3c-a51098a47812-kube-api-access-jscdl\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.684731 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.684667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-model-cache\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.684731 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.684706    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-tmp-dir\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.684861 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.684838    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-home\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.686933 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.686908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-dshm\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.687232 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.687212    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29d5344-6b64-4d26-9b3c-a51098a47812-tls-certs\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.694353 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.694326    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jscdl\" (UniqueName: \"kubernetes.io/projected/e29d5344-6b64-4d26-9b3c-a51098a47812-kube-api-access-jscdl\") pod \"auth-test-model-910fb6-kserve-785f79b775-dhbv5\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") " pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.835174 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.835078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:25.976768 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:25.976736    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"]
Jul 28 14:44:25.978249 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:44:25.978218    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode29d5344_6b64_4d26_9b3c_a51098a47812.slice/crio-e070e3b37adff818fcd981f8a04cbff4125c975e3b4b1420bba608b35490406d WatchSource:0}: Error finding container e070e3b37adff818fcd981f8a04cbff4125c975e3b4b1420bba608b35490406d: Status 404 returned error can't find the container with id e070e3b37adff818fcd981f8a04cbff4125c975e3b4b1420bba608b35490406d
Jul 28 14:44:26.800676 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:26.800637    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5" event={"ID":"e29d5344-6b64-4d26-9b3c-a51098a47812","Type":"ContainerStarted","Data":"707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96"}
Jul 28 14:44:26.800676 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:26.800678    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5" event={"ID":"e29d5344-6b64-4d26-9b3c-a51098a47812","Type":"ContainerStarted","Data":"e070e3b37adff818fcd981f8a04cbff4125c975e3b4b1420bba608b35490406d"}
Jul 28 14:44:26.821341 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:26.821282    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5" podStartSLOduration=1.821267321 podStartE2EDuration="1.821267321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:44:26.819055071 +0000 UTC m=+2682.946445762" watchObservedRunningTime="2026-07-28 14:44:26.821267321 +0000 UTC m=+2682.948658011"
Jul 28 14:44:28.737958 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:28.737924    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:44:35.835365 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:35.835268    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:35.835365 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:35.835319    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:35.848729 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:35.848702    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:35.861792 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:35.861763    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:44:44.510499 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:44.510468    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:44:44.517204 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:44:44.517165    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:45:00.144256 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.144218    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t"]
Jul 28 14:45:00.148510 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.148486    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf"]
Jul 28 14:45:00.148657 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.148621    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t"
Jul 28 14:45:00.151315 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.151293    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:45:00.152464 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.152447    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf"
Jul 28 14:45:00.159191 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.159165    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t"]
Jul 28 14:45:00.161268 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.161251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t"
Jul 28 14:45:00.162676 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.162653    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf"]
Jul 28 14:45:00.171677 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.171653    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf"
Jul 28 14:45:00.318882 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.317002    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t"]
Jul 28 14:45:00.320670 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:45:00.320642    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b103b6c_b3e2_4140_870c_889f590a2c52.slice/crio-966e44c7d30861d399faa5ec1960dac572c3847e66fe4f64aa5bfa3d765e6bad WatchSource:0}: Error finding container 966e44c7d30861d399faa5ec1960dac572c3847e66fe4f64aa5bfa3d765e6bad: Status 404 returned error can't find the container with id 966e44c7d30861d399faa5ec1960dac572c3847e66fe4f64aa5bfa3d765e6bad
Jul 28 14:45:00.347283 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.347259    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf"]
Jul 28 14:45:00.348787 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:45:00.348765    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0807e135_8a65_4961_a701_d033af08784a.slice/crio-d5fcd6f19b8e3349657034b5a8d6e597c5f5f62e8843cb81f82362e4dda46cac WatchSource:0}: Error finding container d5fcd6f19b8e3349657034b5a8d6e597c5f5f62e8843cb81f82362e4dda46cac: Status 404 returned error can't find the container with id d5fcd6f19b8e3349657034b5a8d6e597c5f5f62e8843cb81f82362e4dda46cac
Jul 28 14:45:00.992906 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.992864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf" event={"ID":"0807e135-8a65-4961-a701-d033af08784a","Type":"ContainerStarted","Data":"dea8c3e5b5e0c3cd97c205a0bf0f931a6e213ebf9e60772071e66d1044f08d20"}
Jul 28 14:45:00.993118 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.992922    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf" event={"ID":"0807e135-8a65-4961-a701-d033af08784a","Type":"ContainerStarted","Data":"d5fcd6f19b8e3349657034b5a8d6e597c5f5f62e8843cb81f82362e4dda46cac"}
Jul 28 14:45:00.994394 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.994367    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t" event={"ID":"0b103b6c-b3e2-4140-870c-889f590a2c52","Type":"ContainerStarted","Data":"51e7f76043ee0670926dc5f2754caed6a70bfa5e5a8bb15d21ed1afdebe4e3db"}
Jul 28 14:45:00.994494 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:00.994404    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t" event={"ID":"0b103b6c-b3e2-4140-870c-889f590a2c52","Type":"ContainerStarted","Data":"966e44c7d30861d399faa5ec1960dac572c3847e66fe4f64aa5bfa3d765e6bad"}
Jul 28 14:45:01.010396 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:01.010346    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf" podStartSLOduration=1.010330733 podStartE2EDuration="1.010330733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:45:01.008959192 +0000 UTC m=+2717.136349934" watchObservedRunningTime="2026-07-28 14:45:01.010330733 +0000 UTC m=+2717.137721423"
Jul 28 14:45:01.026417 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:01.026375    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t" podStartSLOduration=1.026360388 podStartE2EDuration="1.026360388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:45:01.024950351 +0000 UTC m=+2717.152341042" watchObservedRunningTime="2026-07-28 14:45:01.026360388 +0000 UTC m=+2717.153751079"
Jul 28 14:45:02.001927 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:02.001891    2579 generic.go:356] "Generic (PLEG): container finished" podID="0807e135-8a65-4961-a701-d033af08784a" containerID="dea8c3e5b5e0c3cd97c205a0bf0f931a6e213ebf9e60772071e66d1044f08d20" exitCode=0
Jul 28 14:45:02.002412 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:02.001971    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf" event={"ID":"0807e135-8a65-4961-a701-d033af08784a","Type":"ContainerDied","Data":"dea8c3e5b5e0c3cd97c205a0bf0f931a6e213ebf9e60772071e66d1044f08d20"}
Jul 28 14:45:02.003511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:02.003489    2579 generic.go:356] "Generic (PLEG): container finished" podID="0b103b6c-b3e2-4140-870c-889f590a2c52" containerID="51e7f76043ee0670926dc5f2754caed6a70bfa5e5a8bb15d21ed1afdebe4e3db" exitCode=0
Jul 28 14:45:02.003616 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:02.003522    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t" event={"ID":"0b103b6c-b3e2-4140-870c-889f590a2c52","Type":"ContainerDied","Data":"51e7f76043ee0670926dc5f2754caed6a70bfa5e5a8bb15d21ed1afdebe4e3db"}
Jul 28 14:45:03.140216 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:03.140192    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf"
Jul 28 14:45:03.176757 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:03.176727    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t"
Jul 28 14:45:04.019147 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:04.019119    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf"
Jul 28 14:45:04.019147 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:04.019132    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754165-zkkbf" event={"ID":"0807e135-8a65-4961-a701-d033af08784a","Type":"ContainerDied","Data":"d5fcd6f19b8e3349657034b5a8d6e597c5f5f62e8843cb81f82362e4dda46cac"}
Jul 28 14:45:04.019400 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:04.019185    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5fcd6f19b8e3349657034b5a8d6e597c5f5f62e8843cb81f82362e4dda46cac"
Jul 28 14:45:04.020550 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:04.020531    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t"
Jul 28 14:45:04.020668 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:04.020575    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t" event={"ID":"0b103b6c-b3e2-4140-870c-889f590a2c52","Type":"ContainerDied","Data":"966e44c7d30861d399faa5ec1960dac572c3847e66fe4f64aa5bfa3d765e6bad"}
Jul 28 14:45:04.020668 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:04.020620    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="966e44c7d30861d399faa5ec1960dac572c3847e66fe4f64aa5bfa3d765e6bad"
Jul 28 14:45:11.141618 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.141584    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"]
Jul 28 14:45:11.142112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.141957    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0807e135-8a65-4961-a701-d033af08784a" containerName="cleanup"
Jul 28 14:45:11.142112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.141967    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0807e135-8a65-4961-a701-d033af08784a" containerName="cleanup"
Jul 28 14:45:11.142112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.141991    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b103b6c-b3e2-4140-870c-889f590a2c52" containerName="cleanup"
Jul 28 14:45:11.142112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.141997    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b103b6c-b3e2-4140-870c-889f590a2c52" containerName="cleanup"
Jul 28 14:45:11.142112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.142087    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0807e135-8a65-4961-a701-d033af08784a" containerName="cleanup"
Jul 28 14:45:11.142112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.142096    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b103b6c-b3e2-4140-870c-889f590a2c52" containerName="cleanup"
Jul 28 14:45:11.144874 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.144859    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.147330 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.147300    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"kube-root-ca.crt\""
Jul 28 14:45:11.147454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.147399    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"openshift-service-ca.crt\""
Jul 28 14:45:11.148174 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.148155    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"auth-test-model-baa216-kserve-self-signed-certs\""
Jul 28 14:45:11.148283 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.148175    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"default-dockercfg-4jtg4\""
Jul 28 14:45:11.154975 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.154954    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"]
Jul 28 14:45:11.289507 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.289466    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-model-cache\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.289715 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.289518    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tls-certs\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.289715 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.289572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tmp-dir\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.289715 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.289662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-home\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.289871 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.289725    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-dshm\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.289871 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.289769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bjpj\" (UniqueName: \"kubernetes.io/projected/d07ccd38-b1ba-4d3d-849d-5a71982f963a-kube-api-access-6bjpj\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.390305 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.390254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bjpj\" (UniqueName: \"kubernetes.io/projected/d07ccd38-b1ba-4d3d-849d-5a71982f963a-kube-api-access-6bjpj\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.390305 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.390309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-model-cache\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.390584 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.390331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tls-certs\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.390584 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.390372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tmp-dir\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.390584 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.390488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-home\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.390584 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.390551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-dshm\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.390836 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.390808    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tmp-dir\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.390911 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.390879    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-model-cache\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.391254 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.391218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-home\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.393243 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.393181    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-dshm\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.393451 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.393430    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tls-certs\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.398720 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.398703    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bjpj\" (UniqueName: \"kubernetes.io/projected/d07ccd38-b1ba-4d3d-849d-5a71982f963a-kube-api-access-6bjpj\") pod \"auth-test-model-baa216-kserve-7b67574d79-wk7zh\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") " pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.454424 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.454387    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:11.604224 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:11.604196    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"]
Jul 28 14:45:11.605763 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:45:11.605738    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd07ccd38_b1ba_4d3d_849d_5a71982f963a.slice/crio-56c0eb1f448b0675d9a5a624d58705c8e44baa0c2dfa9172646220692f20a229 WatchSource:0}: Error finding container 56c0eb1f448b0675d9a5a624d58705c8e44baa0c2dfa9172646220692f20a229: Status 404 returned error can't find the container with id 56c0eb1f448b0675d9a5a624d58705c8e44baa0c2dfa9172646220692f20a229
Jul 28 14:45:12.075352 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:12.075315    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh" event={"ID":"d07ccd38-b1ba-4d3d-849d-5a71982f963a","Type":"ContainerStarted","Data":"1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6"}
Jul 28 14:45:12.075531 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:12.075364    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh" event={"ID":"d07ccd38-b1ba-4d3d-849d-5a71982f963a","Type":"ContainerStarted","Data":"56c0eb1f448b0675d9a5a624d58705c8e44baa0c2dfa9172646220692f20a229"}
Jul 28 14:45:12.096845 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:12.096791    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh" podStartSLOduration=1.09677667 podStartE2EDuration="1.09677667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:45:12.09469654 +0000 UTC m=+2728.222087225" watchObservedRunningTime="2026-07-28 14:45:12.09677667 +0000 UTC m=+2728.224167361"
Jul 28 14:45:15.128602 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:15.128568    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:45:21.455587 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:21.455551    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:21.456067 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:21.455602    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:21.468280 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:21.468252    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:22.142450 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:22.142423    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:45:43.348711 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:45:43.348670    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:46:04.135893 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:46:04.135855    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:46:18.438744 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:46:18.438663    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:46:30.138088 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:46:30.138056    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:46:47.829047 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:46:47.828994    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:47:01.635427 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:47:01.635394    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:47:13.441789 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:47:13.441750    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:47:31.434319 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:47:31.434288    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:47:45.941913 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:47:45.941824    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:47:52.489863 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:47:52.489831    2579 scope.go:117] "RemoveContainer" containerID="09f3123170d33d88ab93c5f88da6e64678c9ba361a6282d778d488c550780823"
Jul 28 14:47:57.534404 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:47:57.534369    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:48:16.133617 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:48:16.133573    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:48:33.547285 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:48:33.547251    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:48:52.511732 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:48:52.511698    2579 scope.go:117] "RemoveContainer" containerID="6cba5c0166e37a96e267917c79ce9b86398808b8a8cabefafc5e7c1d9009dd99"
Jul 28 14:49:01.063012 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:49:01.062970    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:49:19.354844 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:49:19.354806    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:49:44.550197 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:49:44.550172    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:49:44.562119 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:49:44.562090    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:49:46.056667 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:49:46.056628    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:49:52.138907 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:49:52.138863    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"]
Jul 28 14:49:52.534264 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:49:52.534234    2579 scope.go:117] "RemoveContainer" containerID="2fbd6b41e33d0a4c44384c8f957cb60f8a029743f156892d6c3b8838ad4ae253"
Jul 28 14:49:53.146753 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:49:53.146720    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"]
Jul 28 14:50:04.087726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.087690    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"]
Jul 28 14:50:04.091565 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.091544    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.095210 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.095189    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-910fb6\"/\"sub-test-model-910fb6-kserve-self-signed-certs\""
Jul 28 14:50:04.105220 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.104923    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"]
Jul 28 14:50:04.165062 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.164989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9k42\" (UniqueName: \"kubernetes.io/projected/dd3fa370-fb25-4d6c-9760-f56043089466-kube-api-access-x9k42\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.165251 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.165138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-dshm\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.165251 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.165183    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-tmp-dir\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.165251 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.165207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-model-cache\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.165251 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.165222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd3fa370-fb25-4d6c-9760-f56043089466-tls-certs\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.165397 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.165292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-home\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.266112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.266073    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-model-cache\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.266112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.266111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd3fa370-fb25-4d6c-9760-f56043089466-tls-certs\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.266320 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.266191    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-home\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.266320 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.266227    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9k42\" (UniqueName: \"kubernetes.io/projected/dd3fa370-fb25-4d6c-9760-f56043089466-kube-api-access-x9k42\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.266320 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.266288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-dshm\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.266471 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.266342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-tmp-dir\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.266640 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.266601    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-model-cache\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.266712 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.266631    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-home\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.266712 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.266653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-tmp-dir\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.268632 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.268604    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-dshm\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.268829 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.268811    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd3fa370-fb25-4d6c-9760-f56043089466-tls-certs\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.276675 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.276640    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9k42\" (UniqueName: \"kubernetes.io/projected/dd3fa370-fb25-4d6c-9760-f56043089466-kube-api-access-x9k42\") pod \"sub-test-model-910fb6-kserve-766d789899-hgv5s\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") " pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.400976 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.400891    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:04.543606 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.543564    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"]
Jul 28 14:50:04.545346 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:50:04.545317    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd3fa370_fb25_4d6c_9760_f56043089466.slice/crio-2b899b7c12c7705f8ad66b234ead6233bf85a99070a077241699b1c7d96c52ac WatchSource:0}: Error finding container 2b899b7c12c7705f8ad66b234ead6233bf85a99070a077241699b1c7d96c52ac: Status 404 returned error can't find the container with id 2b899b7c12c7705f8ad66b234ead6233bf85a99070a077241699b1c7d96c52ac
Jul 28 14:50:04.751813 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.751697    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s" event={"ID":"dd3fa370-fb25-4d6c-9760-f56043089466","Type":"ContainerStarted","Data":"6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8"}
Jul 28 14:50:04.751813 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.751769    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s" event={"ID":"dd3fa370-fb25-4d6c-9760-f56043089466","Type":"ContainerStarted","Data":"2b899b7c12c7705f8ad66b234ead6233bf85a99070a077241699b1c7d96c52ac"}
Jul 28 14:50:04.772769 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:04.772716    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s" podStartSLOduration=0.77269979 podStartE2EDuration="772.69979ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:50:04.771102383 +0000 UTC m=+3020.898493076" watchObservedRunningTime="2026-07-28 14:50:04.77269979 +0000 UTC m=+3020.900090559"
Jul 28 14:50:05.039934 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:05.039902    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:50:07.159362 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.159317    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5" podUID="e29d5344-6b64-4d26-9b3c-a51098a47812" containerName="main" containerID="cri-o://707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96" gracePeriod=45
Jul 28 14:50:07.422967 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.422904    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:50:07.596576 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.596539    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-home\") pod \"e29d5344-6b64-4d26-9b3c-a51098a47812\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") "
Jul 28 14:50:07.596788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.596631    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-model-cache\") pod \"e29d5344-6b64-4d26-9b3c-a51098a47812\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") "
Jul 28 14:50:07.596788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.596667    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29d5344-6b64-4d26-9b3c-a51098a47812-tls-certs\") pod \"e29d5344-6b64-4d26-9b3c-a51098a47812\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") "
Jul 28 14:50:07.596788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.596700    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-tmp-dir\") pod \"e29d5344-6b64-4d26-9b3c-a51098a47812\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") "
Jul 28 14:50:07.596788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.596727    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jscdl\" (UniqueName: \"kubernetes.io/projected/e29d5344-6b64-4d26-9b3c-a51098a47812-kube-api-access-jscdl\") pod \"e29d5344-6b64-4d26-9b3c-a51098a47812\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") "
Jul 28 14:50:07.596788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.596782    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-dshm\") pod \"e29d5344-6b64-4d26-9b3c-a51098a47812\" (UID: \"e29d5344-6b64-4d26-9b3c-a51098a47812\") "
Jul 28 14:50:07.597085 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.596891    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-home" (OuterVolumeSpecName: "home") pod "e29d5344-6b64-4d26-9b3c-a51098a47812" (UID: "e29d5344-6b64-4d26-9b3c-a51098a47812"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:50:07.597085 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.596919    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-model-cache" (OuterVolumeSpecName: "model-cache") pod "e29d5344-6b64-4d26-9b3c-a51098a47812" (UID: "e29d5344-6b64-4d26-9b3c-a51098a47812"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:50:07.597085 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.597039    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e29d5344-6b64-4d26-9b3c-a51098a47812" (UID: "e29d5344-6b64-4d26-9b3c-a51098a47812"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:50:07.597203 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.597164    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-model-cache\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:07.597203 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.597180    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-tmp-dir\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:07.597203 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.597189    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-home\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:07.599239 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.599218    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-dshm" (OuterVolumeSpecName: "dshm") pod "e29d5344-6b64-4d26-9b3c-a51098a47812" (UID: "e29d5344-6b64-4d26-9b3c-a51098a47812"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:50:07.599239 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.599233    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29d5344-6b64-4d26-9b3c-a51098a47812-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e29d5344-6b64-4d26-9b3c-a51098a47812" (UID: "e29d5344-6b64-4d26-9b3c-a51098a47812"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:50:07.599377 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.599247    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e29d5344-6b64-4d26-9b3c-a51098a47812-kube-api-access-jscdl" (OuterVolumeSpecName: "kube-api-access-jscdl") pod "e29d5344-6b64-4d26-9b3c-a51098a47812" (UID: "e29d5344-6b64-4d26-9b3c-a51098a47812"). InnerVolumeSpecName "kube-api-access-jscdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:50:07.698595 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.698499    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29d5344-6b64-4d26-9b3c-a51098a47812-tls-certs\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:07.698595 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.698542    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jscdl\" (UniqueName: \"kubernetes.io/projected/e29d5344-6b64-4d26-9b3c-a51098a47812-kube-api-access-jscdl\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:07.698595 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.698554    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29d5344-6b64-4d26-9b3c-a51098a47812-dshm\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:07.776137 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.776104    2579 generic.go:356] "Generic (PLEG): container finished" podID="e29d5344-6b64-4d26-9b3c-a51098a47812" containerID="707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96" exitCode=0
Jul 28 14:50:07.776301 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.776158    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"
Jul 28 14:50:07.776301 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.776165    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5" event={"ID":"e29d5344-6b64-4d26-9b3c-a51098a47812","Type":"ContainerDied","Data":"707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96"}
Jul 28 14:50:07.776301 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.776197    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5" event={"ID":"e29d5344-6b64-4d26-9b3c-a51098a47812","Type":"ContainerDied","Data":"e070e3b37adff818fcd981f8a04cbff4125c975e3b4b1420bba608b35490406d"}
Jul 28 14:50:07.776301 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.776214    2579 scope.go:117] "RemoveContainer" containerID="707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96"
Jul 28 14:50:07.786973 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.786954    2579 scope.go:117] "RemoveContainer" containerID="707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96"
Jul 28 14:50:07.787276 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:50:07.787248    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96\": container with ID starting with 707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96 not found: ID does not exist" containerID="707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96"
Jul 28 14:50:07.787363 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.787277    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96"} err="failed to get container status \"707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96\": rpc error: code = NotFound desc = could not find container \"707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96\": container with ID starting with 707a049af9661c31752be4d6ee4bf10e8ce204e8911d7e908e7ce65ced6fae96 not found: ID does not exist"
Jul 28 14:50:07.803191 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.803164    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"]
Jul 28 14:50:07.808281 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:07.808258    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/auth-test-model-910fb6-kserve-785f79b775-dhbv5"]
Jul 28 14:50:08.166520 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.166462    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh" podUID="d07ccd38-b1ba-4d3d-849d-5a71982f963a" containerName="main" containerID="cri-o://1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6" gracePeriod=45
Jul 28 14:50:08.343944 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.343909    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e29d5344-6b64-4d26-9b3c-a51098a47812" path="/var/lib/kubelet/pods/e29d5344-6b64-4d26-9b3c-a51098a47812/volumes"
Jul 28 14:50:08.409672 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.409647    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:50:08.505835 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.505745    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-model-cache\") pod \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") "
Jul 28 14:50:08.505835 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.505782    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-home\") pod \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") "
Jul 28 14:50:08.506198 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.505848    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bjpj\" (UniqueName: \"kubernetes.io/projected/d07ccd38-b1ba-4d3d-849d-5a71982f963a-kube-api-access-6bjpj\") pod \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") "
Jul 28 14:50:08.506198 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.506005    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tmp-dir\") pod \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") "
Jul 28 14:50:08.506198 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.506085    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-dshm\") pod \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") "
Jul 28 14:50:08.506198 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.506104    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-model-cache" (OuterVolumeSpecName: "model-cache") pod "d07ccd38-b1ba-4d3d-849d-5a71982f963a" (UID: "d07ccd38-b1ba-4d3d-849d-5a71982f963a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:50:08.506198 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.506139    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-home" (OuterVolumeSpecName: "home") pod "d07ccd38-b1ba-4d3d-849d-5a71982f963a" (UID: "d07ccd38-b1ba-4d3d-849d-5a71982f963a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:50:08.506198 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.506171    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tls-certs\") pod \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\" (UID: \"d07ccd38-b1ba-4d3d-849d-5a71982f963a\") "
Jul 28 14:50:08.506513 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.506264    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d07ccd38-b1ba-4d3d-849d-5a71982f963a" (UID: "d07ccd38-b1ba-4d3d-849d-5a71982f963a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:50:08.506513 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.506479    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-model-cache\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:08.506513 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.506498    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-home\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:08.506513 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.506511    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tmp-dir\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:08.508371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.508342    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d07ccd38-b1ba-4d3d-849d-5a71982f963a" (UID: "d07ccd38-b1ba-4d3d-849d-5a71982f963a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:50:08.508371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.508345    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d07ccd38-b1ba-4d3d-849d-5a71982f963a-kube-api-access-6bjpj" (OuterVolumeSpecName: "kube-api-access-6bjpj") pod "d07ccd38-b1ba-4d3d-849d-5a71982f963a" (UID: "d07ccd38-b1ba-4d3d-849d-5a71982f963a"). InnerVolumeSpecName "kube-api-access-6bjpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:50:08.508525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.508364    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-dshm" (OuterVolumeSpecName: "dshm") pod "d07ccd38-b1ba-4d3d-849d-5a71982f963a" (UID: "d07ccd38-b1ba-4d3d-849d-5a71982f963a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:50:08.607762 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.607722    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d07ccd38-b1ba-4d3d-849d-5a71982f963a-tls-certs\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:08.607762 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.607760    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bjpj\" (UniqueName: \"kubernetes.io/projected/d07ccd38-b1ba-4d3d-849d-5a71982f963a-kube-api-access-6bjpj\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:08.607973 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.607775    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d07ccd38-b1ba-4d3d-849d-5a71982f963a-dshm\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:50:08.782614 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.782582    2579 generic.go:356] "Generic (PLEG): container finished" podID="d07ccd38-b1ba-4d3d-849d-5a71982f963a" containerID="1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6" exitCode=0
Jul 28 14:50:08.782816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.782628    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"
Jul 28 14:50:08.782816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.782671    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh" event={"ID":"d07ccd38-b1ba-4d3d-849d-5a71982f963a","Type":"ContainerDied","Data":"1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6"}
Jul 28 14:50:08.782816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.782709    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh" event={"ID":"d07ccd38-b1ba-4d3d-849d-5a71982f963a","Type":"ContainerDied","Data":"56c0eb1f448b0675d9a5a624d58705c8e44baa0c2dfa9172646220692f20a229"}
Jul 28 14:50:08.782816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.782726    2579 scope.go:117] "RemoveContainer" containerID="1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6"
Jul 28 14:50:08.792329 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.792313    2579 scope.go:117] "RemoveContainer" containerID="1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6"
Jul 28 14:50:08.792597 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:50:08.792578    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6\": container with ID starting with 1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6 not found: ID does not exist" containerID="1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6"
Jul 28 14:50:08.792658 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.792607    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6"} err="failed to get container status \"1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6\": rpc error: code = NotFound desc = could not find container \"1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6\": container with ID starting with 1b9a42ec73f126d1bd3ebc8cb0ee7f3b6332140485408c06ace7e0cd860ff8d6 not found: ID does not exist"
Jul 28 14:50:08.806007 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.805981    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"]
Jul 28 14:50:08.810784 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:08.810761    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/auth-test-model-baa216-kserve-7b67574d79-wk7zh"]
Jul 28 14:50:10.343175 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:10.343141    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d07ccd38-b1ba-4d3d-849d-5a71982f963a" path="/var/lib/kubelet/pods/d07ccd38-b1ba-4d3d-849d-5a71982f963a/volumes"
Jul 28 14:50:14.401851 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:14.401817    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:14.402277 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:14.401869    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:14.414579 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:14.414549    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:14.827587 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:14.827559    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:50:14.935743 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:14.935703    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:50:23.441059 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.440999    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"]
Jul 28 14:50:23.441610 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.441594    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29d5344-6b64-4d26-9b3c-a51098a47812" containerName="main"
Jul 28 14:50:23.441656 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.441614    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29d5344-6b64-4d26-9b3c-a51098a47812" containerName="main"
Jul 28 14:50:23.441656 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.441645    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d07ccd38-b1ba-4d3d-849d-5a71982f963a" containerName="main"
Jul 28 14:50:23.441656 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.441654    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07ccd38-b1ba-4d3d-849d-5a71982f963a" containerName="main"
Jul 28 14:50:23.441754 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.441746    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29d5344-6b64-4d26-9b3c-a51098a47812" containerName="main"
Jul 28 14:50:23.441788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.441758    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d07ccd38-b1ba-4d3d-849d-5a71982f963a" containerName="main"
Jul 28 14:50:23.445151 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.445131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.448065 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.448011    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"kube-root-ca.crt\""
Jul 28 14:50:23.448831 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.448808    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"sub-test-model-baa216-kserve-self-signed-certs\""
Jul 28 14:50:23.448956 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.448821    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"default-dockercfg-4jtg4\""
Jul 28 14:50:23.449068 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.449047    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"openshift-service-ca.crt\""
Jul 28 14:50:23.456046 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.456006    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"]
Jul 28 14:50:23.535619 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.535584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-dshm\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.535817 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.535645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-model-cache\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.535817 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.535684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-home\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.535817 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.535700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tls-certs\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.535817 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.535719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq9xq\" (UniqueName: \"kubernetes.io/projected/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-kube-api-access-sq9xq\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.535817 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.535796    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tmp-dir\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.636262 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.636225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tmp-dir\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.636460 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.636276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-dshm\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.636460 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.636329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-model-cache\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.636460 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.636376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-home\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.636460 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.636398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tls-certs\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.636460 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.636423    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq9xq\" (UniqueName: \"kubernetes.io/projected/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-kube-api-access-sq9xq\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.636737 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.636700    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tmp-dir\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.636793 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.636728    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-model-cache\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.636793 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.636779    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-home\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.638695 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.638677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-dshm\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.639086 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.639066    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tls-certs\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.646511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.646490    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq9xq\" (UniqueName: \"kubernetes.io/projected/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-kube-api-access-sq9xq\") pod \"sub-test-model-baa216-kserve-c87f8c7d5-mdhh9\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") " pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.755866 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.755781    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:23.904497 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:23.904428    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"]
Jul 28 14:50:23.906807 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:50:23.906780    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff9f7a9f_fe75_4e80_8538_bc5abdd725be.slice/crio-a8b28f598d141c115083349cef57089a2de9ee11ea9317b2ee386d570254d3f2 WatchSource:0}: Error finding container a8b28f598d141c115083349cef57089a2de9ee11ea9317b2ee386d570254d3f2: Status 404 returned error can't find the container with id a8b28f598d141c115083349cef57089a2de9ee11ea9317b2ee386d570254d3f2
Jul 28 14:50:24.877293 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:24.877258    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9" event={"ID":"ff9f7a9f-fe75-4e80-8538-bc5abdd725be","Type":"ContainerStarted","Data":"0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b"}
Jul 28 14:50:24.877730 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:24.877303    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9" event={"ID":"ff9f7a9f-fe75-4e80-8538-bc5abdd725be","Type":"ContainerStarted","Data":"a8b28f598d141c115083349cef57089a2de9ee11ea9317b2ee386d570254d3f2"}
Jul 28 14:50:24.897902 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:24.897825    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9" podStartSLOduration=1.897808893 podStartE2EDuration="1.897808893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:50:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:50:24.896773455 +0000 UTC m=+3041.024164146" watchObservedRunningTime="2026-07-28 14:50:24.897808893 +0000 UTC m=+3041.025199586"
Jul 28 14:50:27.541215 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:27.541180    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:50:33.756314 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:33.756272    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:33.756314 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:33.756323    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:33.769106 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:33.769081    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:33.938672 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:33.938644    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:50:56.948764 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:50:56.948725    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:51:16.647610 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:16.647576    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:51:23.322893 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:23.322854    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"]
Jul 28 14:51:24.244340 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:24.244309    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"]
Jul 28 14:51:35.838762 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:35.838725    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:51:38.342067 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.342002    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s" podUID="dd3fa370-fb25-4d6c-9760-f56043089466" containerName="main" containerID="cri-o://6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8" gracePeriod=45
Jul 28 14:51:38.597366 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.597310    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:51:38.722763 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.722720    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-dshm\") pod \"dd3fa370-fb25-4d6c-9760-f56043089466\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") "
Jul 28 14:51:38.722970 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.722780    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9k42\" (UniqueName: \"kubernetes.io/projected/dd3fa370-fb25-4d6c-9760-f56043089466-kube-api-access-x9k42\") pod \"dd3fa370-fb25-4d6c-9760-f56043089466\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") "
Jul 28 14:51:38.722970 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.722817    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd3fa370-fb25-4d6c-9760-f56043089466-tls-certs\") pod \"dd3fa370-fb25-4d6c-9760-f56043089466\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") "
Jul 28 14:51:38.722970 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.722842    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-model-cache\") pod \"dd3fa370-fb25-4d6c-9760-f56043089466\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") "
Jul 28 14:51:38.722970 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.722891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-tmp-dir\") pod \"dd3fa370-fb25-4d6c-9760-f56043089466\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") "
Jul 28 14:51:38.722970 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.722961    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-home\") pod \"dd3fa370-fb25-4d6c-9760-f56043089466\" (UID: \"dd3fa370-fb25-4d6c-9760-f56043089466\") "
Jul 28 14:51:38.723309 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.723191    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd3fa370-fb25-4d6c-9760-f56043089466" (UID: "dd3fa370-fb25-4d6c-9760-f56043089466"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:51:38.723309 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.723206    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd3fa370-fb25-4d6c-9760-f56043089466" (UID: "dd3fa370-fb25-4d6c-9760-f56043089466"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:51:38.723397 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.723378    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-home" (OuterVolumeSpecName: "home") pod "dd3fa370-fb25-4d6c-9760-f56043089466" (UID: "dd3fa370-fb25-4d6c-9760-f56043089466"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:51:38.723501 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.723479    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-model-cache\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:38.723501 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.723499    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-tmp-dir\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:38.723501 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.723508    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-home\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:38.725271 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.725244    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-dshm" (OuterVolumeSpecName: "dshm") pod "dd3fa370-fb25-4d6c-9760-f56043089466" (UID: "dd3fa370-fb25-4d6c-9760-f56043089466"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:51:38.725271 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.725253    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd3fa370-fb25-4d6c-9760-f56043089466-kube-api-access-x9k42" (OuterVolumeSpecName: "kube-api-access-x9k42") pod "dd3fa370-fb25-4d6c-9760-f56043089466" (UID: "dd3fa370-fb25-4d6c-9760-f56043089466"). InnerVolumeSpecName "kube-api-access-x9k42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:51:38.725394 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.725317    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd3fa370-fb25-4d6c-9760-f56043089466-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd3fa370-fb25-4d6c-9760-f56043089466" (UID: "dd3fa370-fb25-4d6c-9760-f56043089466"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:51:38.823984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.823948    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd3fa370-fb25-4d6c-9760-f56043089466-tls-certs\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:38.823984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.823977    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd3fa370-fb25-4d6c-9760-f56043089466-dshm\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:38.823984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:38.823987    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9k42\" (UniqueName: \"kubernetes.io/projected/dd3fa370-fb25-4d6c-9760-f56043089466-kube-api-access-x9k42\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:39.263969 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.263920    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9" podUID="ff9f7a9f-fe75-4e80-8538-bc5abdd725be" containerName="main" containerID="cri-o://0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b" gracePeriod=45
Jul 28 14:51:39.293612 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.293584    2579 generic.go:356] "Generic (PLEG): container finished" podID="dd3fa370-fb25-4d6c-9760-f56043089466" containerID="6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8" exitCode=0
Jul 28 14:51:39.293721 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.293625    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"
Jul 28 14:51:39.293721 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.293663    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s" event={"ID":"dd3fa370-fb25-4d6c-9760-f56043089466","Type":"ContainerDied","Data":"6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8"}
Jul 28 14:51:39.293721 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.293707    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s" event={"ID":"dd3fa370-fb25-4d6c-9760-f56043089466","Type":"ContainerDied","Data":"2b899b7c12c7705f8ad66b234ead6233bf85a99070a077241699b1c7d96c52ac"}
Jul 28 14:51:39.294186 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.293730    2579 scope.go:117] "RemoveContainer" containerID="6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8"
Jul 28 14:51:39.334421 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.334395    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"]
Jul 28 14:51:39.338877 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.338852    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-910fb6/sub-test-model-910fb6-kserve-766d789899-hgv5s"]
Jul 28 14:51:39.377004 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.376908    2579 scope.go:117] "RemoveContainer" containerID="6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8"
Jul 28 14:51:39.377305 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:51:39.377284    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8\": container with ID starting with 6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8 not found: ID does not exist" containerID="6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8"
Jul 28 14:51:39.377366 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.377320    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8"} err="failed to get container status \"6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8\": rpc error: code = NotFound desc = could not find container \"6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8\": container with ID starting with 6f1d34856e5498e0ba629e7401d7203549051259909c7e6d81c4e146fdd86df8 not found: ID does not exist"
Jul 28 14:51:39.520768 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.520713    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:51:39.629957 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.629929    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tmp-dir\") pod \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") "
Jul 28 14:51:39.630129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.629978    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tls-certs\") pod \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") "
Jul 28 14:51:39.630129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630007    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq9xq\" (UniqueName: \"kubernetes.io/projected/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-kube-api-access-sq9xq\") pod \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") "
Jul 28 14:51:39.630129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630065    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-home\") pod \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") "
Jul 28 14:51:39.630269 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630233    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-dshm\") pod \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") "
Jul 28 14:51:39.630331 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630231    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff9f7a9f-fe75-4e80-8538-bc5abdd725be" (UID: "ff9f7a9f-fe75-4e80-8538-bc5abdd725be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:51:39.630331 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630295    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-home" (OuterVolumeSpecName: "home") pod "ff9f7a9f-fe75-4e80-8538-bc5abdd725be" (UID: "ff9f7a9f-fe75-4e80-8538-bc5abdd725be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:51:39.630331 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630291    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-model-cache\") pod \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\" (UID: \"ff9f7a9f-fe75-4e80-8538-bc5abdd725be\") "
Jul 28 14:51:39.630526 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630505    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff9f7a9f-fe75-4e80-8538-bc5abdd725be" (UID: "ff9f7a9f-fe75-4e80-8538-bc5abdd725be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:51:39.630678 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630652    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-model-cache\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:39.630678 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630669    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tmp-dir\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:39.630983 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.630681    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-home\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:39.632326 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.632303    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-dshm" (OuterVolumeSpecName: "dshm") pod "ff9f7a9f-fe75-4e80-8538-bc5abdd725be" (UID: "ff9f7a9f-fe75-4e80-8538-bc5abdd725be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:51:39.632844 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.632821    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff9f7a9f-fe75-4e80-8538-bc5abdd725be" (UID: "ff9f7a9f-fe75-4e80-8538-bc5abdd725be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:51:39.632844 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.632834    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-kube-api-access-sq9xq" (OuterVolumeSpecName: "kube-api-access-sq9xq") pod "ff9f7a9f-fe75-4e80-8538-bc5abdd725be" (UID: "ff9f7a9f-fe75-4e80-8538-bc5abdd725be"). InnerVolumeSpecName "kube-api-access-sq9xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:51:39.731264 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.731231    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq9xq\" (UniqueName: \"kubernetes.io/projected/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-kube-api-access-sq9xq\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:39.731264 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.731258    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-dshm\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:39.731264 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:39.731270    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9f7a9f-fe75-4e80-8538-bc5abdd725be-tls-certs\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:51:40.301935 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.301897    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff9f7a9f-fe75-4e80-8538-bc5abdd725be" containerID="0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b" exitCode=0
Jul 28 14:51:40.302103 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.301960    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"
Jul 28 14:51:40.302103 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.301964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9" event={"ID":"ff9f7a9f-fe75-4e80-8538-bc5abdd725be","Type":"ContainerDied","Data":"0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b"}
Jul 28 14:51:40.302103 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.302001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9" event={"ID":"ff9f7a9f-fe75-4e80-8538-bc5abdd725be","Type":"ContainerDied","Data":"a8b28f598d141c115083349cef57089a2de9ee11ea9317b2ee386d570254d3f2"}
Jul 28 14:51:40.302103 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.302044    2579 scope.go:117] "RemoveContainer" containerID="0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b"
Jul 28 14:51:40.311822 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.311804    2579 scope.go:117] "RemoveContainer" containerID="0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b"
Jul 28 14:51:40.312158 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:51:40.312135    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b\": container with ID starting with 0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b not found: ID does not exist" containerID="0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b"
Jul 28 14:51:40.312263 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.312165    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b"} err="failed to get container status \"0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b\": rpc error: code = NotFound desc = could not find container \"0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b\": container with ID starting with 0ed8c75f07c7ef9097c82b0324f3a826eb36e07426eaf60dcaeac87296cb831b not found: ID does not exist"
Jul 28 14:51:40.327727 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.327702    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"]
Jul 28 14:51:40.332892 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.332875    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/sub-test-model-baa216-kserve-c87f8c7d5-mdhh9"]
Jul 28 14:51:40.342303 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.342282    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd3fa370-fb25-4d6c-9760-f56043089466" path="/var/lib/kubelet/pods/dd3fa370-fb25-4d6c-9760-f56043089466/volumes"
Jul 28 14:51:40.342655 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:51:40.342643    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9f7a9f-fe75-4e80-8538-bc5abdd725be" path="/var/lib/kubelet/pods/ff9f7a9f-fe75-4e80-8538-bc5abdd725be/volumes"
Jul 28 14:52:23.538788 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.538752    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"]
Jul 28 14:52:23.539235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.539173    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9f7a9f-fe75-4e80-8538-bc5abdd725be" containerName="main"
Jul 28 14:52:23.539235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.539186    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9f7a9f-fe75-4e80-8538-bc5abdd725be" containerName="main"
Jul 28 14:52:23.539235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.539206    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd3fa370-fb25-4d6c-9760-f56043089466" containerName="main"
Jul 28 14:52:23.539235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.539211    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd3fa370-fb25-4d6c-9760-f56043089466" containerName="main"
Jul 28 14:52:23.539368 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.539268    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd3fa370-fb25-4d6c-9760-f56043089466" containerName="main"
Jul 28 14:52:23.539368 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.539282    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9f7a9f-fe75-4e80-8538-bc5abdd725be" containerName="main"
Jul 28 14:52:23.542425 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.542404    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.546857 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.546834    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"default-dockercfg-4jtg4\""
Jul 28 14:52:23.546857 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.546851    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"kube-root-ca.crt\""
Jul 28 14:52:23.547081 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.546838    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"openshift-service-ca.crt\""
Jul 28 14:52:23.547081 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.546838    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-baa216\"/\"rate-test-model-baa216-kserve-self-signed-certs\""
Jul 28 14:52:23.555244 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.555222    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"]
Jul 28 14:52:23.630063 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.630003    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-home\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.630249 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.630142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-model-cache\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.630249 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.630175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-dshm\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.630249 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.630204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tmp-dir\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.630429 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.630251    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86qgg\" (UniqueName: \"kubernetes.io/projected/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-kube-api-access-86qgg\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.630429 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.630312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tls-certs\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.731227 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.731193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tls-certs\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.731382 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.731236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-home\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.731382 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.731315    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-model-cache\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.731382 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.731351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-dshm\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.731382 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.731373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tmp-dir\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.731599 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.731410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86qgg\" (UniqueName: \"kubernetes.io/projected/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-kube-api-access-86qgg\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.731732 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.731694    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-home\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.731732 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.731730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tmp-dir\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.731877 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.731837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-model-cache\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.733627 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.733606    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-dshm\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.733948 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.733927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tls-certs\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.740354 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.740333    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86qgg\" (UniqueName: \"kubernetes.io/projected/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-kube-api-access-86qgg\") pod \"rate-test-model-baa216-kserve-64fc6cd95-7h526\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") " pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.852112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.851996    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:23.991561 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:23.991537    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"]
Jul 28 14:52:23.994176 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:52:23.994149    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28ca0dd1_14e6_4d63_a48f_c50b5d1cab4e.slice/crio-42f038b52e95c1d1739d486377d40f51a7791f46fc47dc5a847343d970860e08 WatchSource:0}: Error finding container 42f038b52e95c1d1739d486377d40f51a7791f46fc47dc5a847343d970860e08: Status 404 returned error can't find the container with id 42f038b52e95c1d1739d486377d40f51a7791f46fc47dc5a847343d970860e08
Jul 28 14:52:24.552490 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:24.552456    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526" event={"ID":"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e","Type":"ContainerStarted","Data":"fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7"}
Jul 28 14:52:24.552490 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:24.552495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526" event={"ID":"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e","Type":"ContainerStarted","Data":"42f038b52e95c1d1739d486377d40f51a7791f46fc47dc5a847343d970860e08"}
Jul 28 14:52:24.572248 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:24.572197    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526" podStartSLOduration=1.572181933 podStartE2EDuration="1.572181933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:52:24.570429662 +0000 UTC m=+3160.697820404" watchObservedRunningTime="2026-07-28 14:52:24.572181933 +0000 UTC m=+3160.699572627"
Jul 28 14:52:27.231524 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:27.231486    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:52:33.852435 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:33.852390    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:33.852856 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:33.852455    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:33.865189 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:33.865166    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:34.618148 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:34.618120    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:52:42.073633 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.073592    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hnfgl"]
Jul 28 14:52:42.077671 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.077650    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:42.080059 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.080011    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-nvrz6\""
Jul 28 14:52:42.087615 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.087588    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hnfgl"]
Jul 28 14:52:42.204922 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.204881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hw69\" (UniqueName: \"kubernetes.io/projected/4e6cdac6-aa26-44c6-9704-a985e428e07d-kube-api-access-2hw69\") pod \"kuadrant-operator-catalog-hnfgl\" (UID: \"4e6cdac6-aa26-44c6-9704-a985e428e07d\") " pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:42.305654 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.305616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hw69\" (UniqueName: \"kubernetes.io/projected/4e6cdac6-aa26-44c6-9704-a985e428e07d-kube-api-access-2hw69\") pod \"kuadrant-operator-catalog-hnfgl\" (UID: \"4e6cdac6-aa26-44c6-9704-a985e428e07d\") " pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:42.316294 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.316267    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hw69\" (UniqueName: \"kubernetes.io/projected/4e6cdac6-aa26-44c6-9704-a985e428e07d-kube-api-access-2hw69\") pod \"kuadrant-operator-catalog-hnfgl\" (UID: \"4e6cdac6-aa26-44c6-9704-a985e428e07d\") " pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:42.389689 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.389594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:42.545568 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.545538    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hnfgl"]
Jul 28 14:52:42.547297 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:52:42.547270    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e6cdac6_aa26_44c6_9704_a985e428e07d.slice/crio-035c21365d17af3db943af05af720bef881f6a3469d82acf575b47d3ad51b78a WatchSource:0}: Error finding container 035c21365d17af3db943af05af720bef881f6a3469d82acf575b47d3ad51b78a: Status 404 returned error can't find the container with id 035c21365d17af3db943af05af720bef881f6a3469d82acf575b47d3ad51b78a
Jul 28 14:52:42.549299 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.549278    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:52:42.652553 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:42.652462    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl" event={"ID":"4e6cdac6-aa26-44c6-9704-a985e428e07d","Type":"ContainerStarted","Data":"035c21365d17af3db943af05af720bef881f6a3469d82acf575b47d3ad51b78a"}
Jul 28 14:52:45.674460 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:45.674416    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl" event={"ID":"4e6cdac6-aa26-44c6-9704-a985e428e07d","Type":"ContainerStarted","Data":"f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc"}
Jul 28 14:52:52.390675 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:52.390637    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:52.390675 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:52.390685    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:52.413061 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:52.413034    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:52.429460 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:52.429405    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl" podStartSLOduration=8.242183135 podStartE2EDuration="10.429386493s" totalImagesPullingTime="2.187203358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:52:45.692402156 +0000 UTC m=+3181.819792848" watchObservedRunningTime="2026-07-28 14:52:52.429386493 +0000 UTC m=+3188.556777213"
Jul 28 14:52:52.736257 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:52.736185    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:52.777715 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:52.777682    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hnfgl"]
Jul 28 14:52:54.724081 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:54.723998    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl" podUID="4e6cdac6-aa26-44c6-9704-a985e428e07d" containerName="registry-server" containerID="cri-o://f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc" gracePeriod=2
Jul 28 14:52:54.983247 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:54.983188    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:55.005184 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.005150    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hw69\" (UniqueName: \"kubernetes.io/projected/4e6cdac6-aa26-44c6-9704-a985e428e07d-kube-api-access-2hw69\") pod \"4e6cdac6-aa26-44c6-9704-a985e428e07d\" (UID: \"4e6cdac6-aa26-44c6-9704-a985e428e07d\") "
Jul 28 14:52:55.007374 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.007345    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e6cdac6-aa26-44c6-9704-a985e428e07d-kube-api-access-2hw69" (OuterVolumeSpecName: "kube-api-access-2hw69") pod "4e6cdac6-aa26-44c6-9704-a985e428e07d" (UID: "4e6cdac6-aa26-44c6-9704-a985e428e07d"). InnerVolumeSpecName "kube-api-access-2hw69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:52:55.105897 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.105866    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hw69\" (UniqueName: \"kubernetes.io/projected/4e6cdac6-aa26-44c6-9704-a985e428e07d-kube-api-access-2hw69\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:52:55.730112 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.730080    2579 generic.go:356] "Generic (PLEG): container finished" podID="4e6cdac6-aa26-44c6-9704-a985e428e07d" containerID="f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc" exitCode=0
Jul 28 14:52:55.730593 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.730128    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl"
Jul 28 14:52:55.730593 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.730171    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl" event={"ID":"4e6cdac6-aa26-44c6-9704-a985e428e07d","Type":"ContainerDied","Data":"f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc"}
Jul 28 14:52:55.730593 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.730229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hnfgl" event={"ID":"4e6cdac6-aa26-44c6-9704-a985e428e07d","Type":"ContainerDied","Data":"035c21365d17af3db943af05af720bef881f6a3469d82acf575b47d3ad51b78a"}
Jul 28 14:52:55.730593 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.730261    2579 scope.go:117] "RemoveContainer" containerID="f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc"
Jul 28 14:52:55.740210 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.740195    2579 scope.go:117] "RemoveContainer" containerID="f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc"
Jul 28 14:52:55.740438 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:52:55.740422    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc\": container with ID starting with f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc not found: ID does not exist" containerID="f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc"
Jul 28 14:52:55.740482 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.740446    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc"} err="failed to get container status \"f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc\": rpc error: code = NotFound desc = could not find container \"f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc\": container with ID starting with f8eb9294f1c0101637f276e7cbebbea8e21c259d4ef90703d56acbd57f7d1fcc not found: ID does not exist"
Jul 28 14:52:55.754263 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.754233    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hnfgl"]
Jul 28 14:52:55.760035 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:55.759988    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hnfgl"]
Jul 28 14:52:56.341848 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:56.341812    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e6cdac6-aa26-44c6-9704-a985e428e07d" path="/var/lib/kubelet/pods/4e6cdac6-aa26-44c6-9704-a985e428e07d/volumes"
Jul 28 14:52:58.241132 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:52:58.241096    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:53:12.358841 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:12.358805    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:53:27.140946 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:27.140911    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:53:32.519099 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:32.519066    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"]
Jul 28 14:53:33.561107 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.561071    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7201-140378772920752"]
Jul 28 14:53:33.561494 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.561456    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e6cdac6-aa26-44c6-9704-a985e428e07d" containerName="registry-server"
Jul 28 14:53:33.561494 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.561467    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e6cdac6-aa26-44c6-9704-a985e428e07d" containerName="registry-server"
Jul 28 14:53:33.561573 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.561533    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e6cdac6-aa26-44c6-9704-a985e428e07d" containerName="registry-server"
Jul 28 14:53:33.564743 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.564729    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772920752"
Jul 28 14:53:33.567492 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.567467    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-k9hrz\""
Jul 28 14:53:33.570159 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.570137    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772920752"]
Jul 28 14:53:33.634968 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.634937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shvnk\" (UniqueName: \"kubernetes.io/projected/92259558-d2cd-4844-8fde-ab067f638a1e-kube-api-access-shvnk\") pod \"test-curl-7201-140378772920752\" (UID: \"92259558-d2cd-4844-8fde-ab067f638a1e\") " pod="opendatahub/test-curl-7201-140378772920752"
Jul 28 14:53:33.736302 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.736266    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shvnk\" (UniqueName: \"kubernetes.io/projected/92259558-d2cd-4844-8fde-ab067f638a1e-kube-api-access-shvnk\") pod \"test-curl-7201-140378772920752\" (UID: \"92259558-d2cd-4844-8fde-ab067f638a1e\") " pod="opendatahub/test-curl-7201-140378772920752"
Jul 28 14:53:33.745987 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.745960    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shvnk\" (UniqueName: \"kubernetes.io/projected/92259558-d2cd-4844-8fde-ab067f638a1e-kube-api-access-shvnk\") pod \"test-curl-7201-140378772920752\" (UID: \"92259558-d2cd-4844-8fde-ab067f638a1e\") " pod="opendatahub/test-curl-7201-140378772920752"
Jul 28 14:53:33.875308 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:33.875243    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772920752"
Jul 28 14:53:34.008904 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:34.008882    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772920752"]
Jul 28 14:53:34.010734 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:53:34.010706    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92259558_d2cd_4844_8fde_ab067f638a1e.slice/crio-b6be825975cab7a6b51a0129e492b50f2e58b4d3471cfbea05c3833c1cb45bd5 WatchSource:0}: Error finding container b6be825975cab7a6b51a0129e492b50f2e58b4d3471cfbea05c3833c1cb45bd5: Status 404 returned error can't find the container with id b6be825975cab7a6b51a0129e492b50f2e58b4d3471cfbea05c3833c1cb45bd5
Jul 28 14:53:34.952819 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:34.952786    2579 generic.go:356] "Generic (PLEG): container finished" podID="92259558-d2cd-4844-8fde-ab067f638a1e" containerID="afb6c95c3312206e53b73cfcf9738acb9a0e54b68afe888a2f99f5777fd40725" exitCode=0
Jul 28 14:53:34.953201 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:34.952876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772920752" event={"ID":"92259558-d2cd-4844-8fde-ab067f638a1e","Type":"ContainerDied","Data":"afb6c95c3312206e53b73cfcf9738acb9a0e54b68afe888a2f99f5777fd40725"}
Jul 28 14:53:34.953201 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:34.952916    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772920752" event={"ID":"92259558-d2cd-4844-8fde-ab067f638a1e","Type":"ContainerStarted","Data":"b6be825975cab7a6b51a0129e492b50f2e58b4d3471cfbea05c3833c1cb45bd5"}
Jul 28 14:53:36.095627 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.095598    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772920752"
Jul 28 14:53:36.133149 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.133128    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7201-140378772920752_92259558-d2cd-4844-8fde-ab067f638a1e/test-curl-7201-140378772920752/0.log"
Jul 28 14:53:36.158290 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.158229    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7201-140378772920752"]
Jul 28 14:53:36.160528 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.160508    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7201-140378772920752"]
Jul 28 14:53:36.260689 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.260668    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shvnk\" (UniqueName: \"kubernetes.io/projected/92259558-d2cd-4844-8fde-ab067f638a1e-kube-api-access-shvnk\") pod \"92259558-d2cd-4844-8fde-ab067f638a1e\" (UID: \"92259558-d2cd-4844-8fde-ab067f638a1e\") "
Jul 28 14:53:36.262938 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.262920    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92259558-d2cd-4844-8fde-ab067f638a1e-kube-api-access-shvnk" (OuterVolumeSpecName: "kube-api-access-shvnk") pod "92259558-d2cd-4844-8fde-ab067f638a1e" (UID: "92259558-d2cd-4844-8fde-ab067f638a1e"). InnerVolumeSpecName "kube-api-access-shvnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:53:36.296486 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.296459    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7201-140378772907440"]
Jul 28 14:53:36.296894 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.296875    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92259558-d2cd-4844-8fde-ab067f638a1e" containerName="test-curl-7201-140378772920752"
Jul 28 14:53:36.296894 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.296894    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="92259558-d2cd-4844-8fde-ab067f638a1e" containerName="test-curl-7201-140378772920752"
Jul 28 14:53:36.297006 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.296976    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="92259558-d2cd-4844-8fde-ab067f638a1e" containerName="test-curl-7201-140378772920752"
Jul 28 14:53:36.300075 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.300060    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772907440"
Jul 28 14:53:36.308238 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.308147    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772907440"]
Jul 28 14:53:36.348909 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.348875    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92259558-d2cd-4844-8fde-ab067f638a1e" path="/var/lib/kubelet/pods/92259558-d2cd-4844-8fde-ab067f638a1e/volumes"
Jul 28 14:53:36.362284 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.362263    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shvnk\" (UniqueName: \"kubernetes.io/projected/92259558-d2cd-4844-8fde-ab067f638a1e-kube-api-access-shvnk\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:36.463568 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.463535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdcgh\" (UniqueName: \"kubernetes.io/projected/d2481016-95b2-4fe6-90b2-aabd902e7bc8-kube-api-access-qdcgh\") pod \"test-curl-7201-140378772907440\" (UID: \"d2481016-95b2-4fe6-90b2-aabd902e7bc8\") " pod="opendatahub/test-curl-7201-140378772907440"
Jul 28 14:53:36.564945 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.564872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdcgh\" (UniqueName: \"kubernetes.io/projected/d2481016-95b2-4fe6-90b2-aabd902e7bc8-kube-api-access-qdcgh\") pod \"test-curl-7201-140378772907440\" (UID: \"d2481016-95b2-4fe6-90b2-aabd902e7bc8\") " pod="opendatahub/test-curl-7201-140378772907440"
Jul 28 14:53:36.574236 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.574212    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdcgh\" (UniqueName: \"kubernetes.io/projected/d2481016-95b2-4fe6-90b2-aabd902e7bc8-kube-api-access-qdcgh\") pod \"test-curl-7201-140378772907440\" (UID: \"d2481016-95b2-4fe6-90b2-aabd902e7bc8\") " pod="opendatahub/test-curl-7201-140378772907440"
Jul 28 14:53:36.611080 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.611059    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772907440"
Jul 28 14:53:36.747182 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.747155    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772907440"]
Jul 28 14:53:36.747874 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:53:36.747854    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2481016_95b2_4fe6_90b2_aabd902e7bc8.slice/crio-d2bc122b7ba5d93f42df2e22dc9e31e99d7e2b32402becddd3ab28ba513a39d0 WatchSource:0}: Error finding container d2bc122b7ba5d93f42df2e22dc9e31e99d7e2b32402becddd3ab28ba513a39d0: Status 404 returned error can't find the container with id d2bc122b7ba5d93f42df2e22dc9e31e99d7e2b32402becddd3ab28ba513a39d0
Jul 28 14:53:36.965312 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.965224    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772920752"
Jul 28 14:53:36.965312 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.965217    2579 scope.go:117] "RemoveContainer" containerID="afb6c95c3312206e53b73cfcf9738acb9a0e54b68afe888a2f99f5777fd40725"
Jul 28 14:53:36.966669 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:36.966424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772907440" event={"ID":"d2481016-95b2-4fe6-90b2-aabd902e7bc8","Type":"ContainerStarted","Data":"d2bc122b7ba5d93f42df2e22dc9e31e99d7e2b32402becddd3ab28ba513a39d0"}
Jul 28 14:53:37.974421 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:37.974387    2579 generic.go:356] "Generic (PLEG): container finished" podID="d2481016-95b2-4fe6-90b2-aabd902e7bc8" containerID="7b365ca22cde388a5586b73cc1a73f029f412dcf422d0600080888216c34bd92" exitCode=0
Jul 28 14:53:37.974805 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:37.974459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772907440" event={"ID":"d2481016-95b2-4fe6-90b2-aabd902e7bc8","Type":"ContainerDied","Data":"7b365ca22cde388a5586b73cc1a73f029f412dcf422d0600080888216c34bd92"}
Jul 28 14:53:39.116147 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.116124    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772907440"
Jul 28 14:53:39.136873 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.136855    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7201-140378772907440_d2481016-95b2-4fe6-90b2-aabd902e7bc8/test-curl-7201-140378772907440/0.log"
Jul 28 14:53:39.162923 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.162893    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7201-140378772907440"]
Jul 28 14:53:39.167081 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.167052    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7201-140378772907440"]
Jul 28 14:53:39.288656 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.288633    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdcgh\" (UniqueName: \"kubernetes.io/projected/d2481016-95b2-4fe6-90b2-aabd902e7bc8-kube-api-access-qdcgh\") pod \"d2481016-95b2-4fe6-90b2-aabd902e7bc8\" (UID: \"d2481016-95b2-4fe6-90b2-aabd902e7bc8\") "
Jul 28 14:53:39.290918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.290894    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2481016-95b2-4fe6-90b2-aabd902e7bc8-kube-api-access-qdcgh" (OuterVolumeSpecName: "kube-api-access-qdcgh") pod "d2481016-95b2-4fe6-90b2-aabd902e7bc8" (UID: "d2481016-95b2-4fe6-90b2-aabd902e7bc8"). InnerVolumeSpecName "kube-api-access-qdcgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:53:39.348364 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.348338    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:39.348728 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.348715    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2481016-95b2-4fe6-90b2-aabd902e7bc8" containerName="test-curl-7201-140378772907440"
Jul 28 14:53:39.348768 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.348729    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2481016-95b2-4fe6-90b2-aabd902e7bc8" containerName="test-curl-7201-140378772907440"
Jul 28 14:53:39.348816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.348806    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2481016-95b2-4fe6-90b2-aabd902e7bc8" containerName="test-curl-7201-140378772907440"
Jul 28 14:53:39.351918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.351902    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:39.357261 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.357227    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:39.389909 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.389886    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdcgh\" (UniqueName: \"kubernetes.io/projected/d2481016-95b2-4fe6-90b2-aabd902e7bc8-kube-api-access-qdcgh\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:39.491211 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.491184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g6vv\" (UniqueName: \"kubernetes.io/projected/56a64eb6-430c-4086-a29f-3fb4d0c4ff19-kube-api-access-5g6vv\") pod \"test-curl-7201-140378772936720\" (UID: \"56a64eb6-430c-4086-a29f-3fb4d0c4ff19\") " pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:39.592413 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.592346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g6vv\" (UniqueName: \"kubernetes.io/projected/56a64eb6-430c-4086-a29f-3fb4d0c4ff19-kube-api-access-5g6vv\") pod \"test-curl-7201-140378772936720\" (UID: \"56a64eb6-430c-4086-a29f-3fb4d0c4ff19\") " pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:39.602382 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.602351    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g6vv\" (UniqueName: \"kubernetes.io/projected/56a64eb6-430c-4086-a29f-3fb4d0c4ff19-kube-api-access-5g6vv\") pod \"test-curl-7201-140378772936720\" (UID: \"56a64eb6-430c-4086-a29f-3fb4d0c4ff19\") " pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:39.662731 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.662709    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:39.794645 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.794614    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:39.795962 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:53:39.795940    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56a64eb6_430c_4086_a29f_3fb4d0c4ff19.slice/crio-0bb9e67970ce68b716b862c4f91cac13ae6bc787ed450ecf70fadd9199c2d617 WatchSource:0}: Error finding container 0bb9e67970ce68b716b862c4f91cac13ae6bc787ed450ecf70fadd9199c2d617: Status 404 returned error can't find the container with id 0bb9e67970ce68b716b862c4f91cac13ae6bc787ed450ecf70fadd9199c2d617
Jul 28 14:53:39.987152 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.987067    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772936720" event={"ID":"56a64eb6-430c-4086-a29f-3fb4d0c4ff19","Type":"ContainerStarted","Data":"0bb9e67970ce68b716b862c4f91cac13ae6bc787ed450ecf70fadd9199c2d617"}
Jul 28 14:53:39.988379 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.988356    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772907440"
Jul 28 14:53:39.988484 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:39.988367    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2bc122b7ba5d93f42df2e22dc9e31e99d7e2b32402becddd3ab28ba513a39d0"
Jul 28 14:53:39.998418 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:53:39.998393    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7201-140378772907440\" is forbidden: User \"system:node:ip-10-0-143-3.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-3.ec2.internal' and this object" podUID="d2481016-95b2-4fe6-90b2-aabd902e7bc8" pod="opendatahub/test-curl-7201-140378772907440"
Jul 28 14:53:40.341639 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:40.341608    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2481016-95b2-4fe6-90b2-aabd902e7bc8" path="/var/lib/kubelet/pods/d2481016-95b2-4fe6-90b2-aabd902e7bc8/volumes"
Jul 28 14:53:40.995258 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:40.995229    2579 generic.go:356] "Generic (PLEG): container finished" podID="56a64eb6-430c-4086-a29f-3fb4d0c4ff19" containerID="6ba2a152d2926af05bb515f03476c768b49db204a48f44f1232a393b37098cad" exitCode=0
Jul 28 14:53:40.995438 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:40.995318    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772936720" event={"ID":"56a64eb6-430c-4086-a29f-3fb4d0c4ff19","Type":"ContainerDied","Data":"6ba2a152d2926af05bb515f03476c768b49db204a48f44f1232a393b37098cad"}
Jul 28 14:53:42.143243 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.143217    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:42.165409 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.165383    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7201-140378772936720_56a64eb6-430c-4086-a29f-3fb4d0c4ff19/test-curl-7201-140378772936720/0.log"
Jul 28 14:53:42.189553 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.189522    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:42.193785 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.193758    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:42.318245 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.318195    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g6vv\" (UniqueName: \"kubernetes.io/projected/56a64eb6-430c-4086-a29f-3fb4d0c4ff19-kube-api-access-5g6vv\") pod \"56a64eb6-430c-4086-a29f-3fb4d0c4ff19\" (UID: \"56a64eb6-430c-4086-a29f-3fb4d0c4ff19\") "
Jul 28 14:53:42.320372 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.320344    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:42.320490 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.320380    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56a64eb6-430c-4086-a29f-3fb4d0c4ff19-kube-api-access-5g6vv" (OuterVolumeSpecName: "kube-api-access-5g6vv") pod "56a64eb6-430c-4086-a29f-3fb4d0c4ff19" (UID: "56a64eb6-430c-4086-a29f-3fb4d0c4ff19"). InnerVolumeSpecName "kube-api-access-5g6vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:53:42.320747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.320734    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a64eb6-430c-4086-a29f-3fb4d0c4ff19" containerName="test-curl-7201-140378772936720"
Jul 28 14:53:42.320783 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.320748    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a64eb6-430c-4086-a29f-3fb4d0c4ff19" containerName="test-curl-7201-140378772936720"
Jul 28 14:53:42.320829 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.320819    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a64eb6-430c-4086-a29f-3fb4d0c4ff19" containerName="test-curl-7201-140378772936720"
Jul 28 14:53:42.323885 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.323871    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:42.331753 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.331727    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:42.341938 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.341917    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56a64eb6-430c-4086-a29f-3fb4d0c4ff19" path="/var/lib/kubelet/pods/56a64eb6-430c-4086-a29f-3fb4d0c4ff19/volumes"
Jul 28 14:53:42.419672 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.419640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrlrg\" (UniqueName: \"kubernetes.io/projected/d4c81d7c-bebb-4834-9016-2e2cffcb7f5d-kube-api-access-mrlrg\") pod \"test-curl-7201-140378772936720\" (UID: \"d4c81d7c-bebb-4834-9016-2e2cffcb7f5d\") " pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:42.419829 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.419743    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g6vv\" (UniqueName: \"kubernetes.io/projected/56a64eb6-430c-4086-a29f-3fb4d0c4ff19-kube-api-access-5g6vv\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:42.521124 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.521085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrlrg\" (UniqueName: \"kubernetes.io/projected/d4c81d7c-bebb-4834-9016-2e2cffcb7f5d-kube-api-access-mrlrg\") pod \"test-curl-7201-140378772936720\" (UID: \"d4c81d7c-bebb-4834-9016-2e2cffcb7f5d\") " pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:42.529783 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.529760    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrlrg\" (UniqueName: \"kubernetes.io/projected/d4c81d7c-bebb-4834-9016-2e2cffcb7f5d-kube-api-access-mrlrg\") pod \"test-curl-7201-140378772936720\" (UID: \"d4c81d7c-bebb-4834-9016-2e2cffcb7f5d\") " pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:42.637882 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.637801    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:42.771524 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:42.771492    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:42.773286 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:53:42.773252    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4c81d7c_bebb_4834_9016_2e2cffcb7f5d.slice/crio-2c3ce2e36441a5ef424c4431fc3fcbebe3a42d1286b5f3a8021611a087197683 WatchSource:0}: Error finding container 2c3ce2e36441a5ef424c4431fc3fcbebe3a42d1286b5f3a8021611a087197683: Status 404 returned error can't find the container with id 2c3ce2e36441a5ef424c4431fc3fcbebe3a42d1286b5f3a8021611a087197683
Jul 28 14:53:43.007445 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:43.007413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772936720" event={"ID":"d4c81d7c-bebb-4834-9016-2e2cffcb7f5d","Type":"ContainerStarted","Data":"2c3ce2e36441a5ef424c4431fc3fcbebe3a42d1286b5f3a8021611a087197683"}
Jul 28 14:53:43.008722 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:43.008704    2579 scope.go:117] "RemoveContainer" containerID="6ba2a152d2926af05bb515f03476c768b49db204a48f44f1232a393b37098cad"
Jul 28 14:53:43.008845 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:43.008705    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:44.016908 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:44.016874    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4c81d7c-bebb-4834-9016-2e2cffcb7f5d" containerID="9b8f9dd0d46602977559b0e7e174002b3d073c1be689f1ee68847d9daae5ac61" exitCode=0
Jul 28 14:53:44.017295 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:44.016929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772936720" event={"ID":"d4c81d7c-bebb-4834-9016-2e2cffcb7f5d","Type":"ContainerDied","Data":"9b8f9dd0d46602977559b0e7e174002b3d073c1be689f1ee68847d9daae5ac61"}
Jul 28 14:53:44.736769 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:44.736735    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:53:45.163317 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.163290    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:45.187294 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.187261    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7201-140378772936720_d4c81d7c-bebb-4834-9016-2e2cffcb7f5d/test-curl-7201-140378772936720/0.log"
Jul 28 14:53:45.213453 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.213425    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:45.221180 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.221153    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7201-140378772936720"]
Jul 28 14:53:45.345438 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.345358    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrlrg\" (UniqueName: \"kubernetes.io/projected/d4c81d7c-bebb-4834-9016-2e2cffcb7f5d-kube-api-access-mrlrg\") pod \"d4c81d7c-bebb-4834-9016-2e2cffcb7f5d\" (UID: \"d4c81d7c-bebb-4834-9016-2e2cffcb7f5d\") "
Jul 28 14:53:45.347543 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.347513    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c81d7c-bebb-4834-9016-2e2cffcb7f5d-kube-api-access-mrlrg" (OuterVolumeSpecName: "kube-api-access-mrlrg") pod "d4c81d7c-bebb-4834-9016-2e2cffcb7f5d" (UID: "d4c81d7c-bebb-4834-9016-2e2cffcb7f5d"). InnerVolumeSpecName "kube-api-access-mrlrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:53:45.362206 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.362182    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:45.362589 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.362576    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4c81d7c-bebb-4834-9016-2e2cffcb7f5d" containerName="test-curl-7201-140378772936720"
Jul 28 14:53:45.362629 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.362591    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c81d7c-bebb-4834-9016-2e2cffcb7f5d" containerName="test-curl-7201-140378772936720"
Jul 28 14:53:45.362681 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.362671    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4c81d7c-bebb-4834-9016-2e2cffcb7f5d" containerName="test-curl-7201-140378772936720"
Jul 28 14:53:45.365641 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.365625    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:45.373645 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.373619    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:45.446033 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.445998    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrlrg\" (UniqueName: \"kubernetes.io/projected/d4c81d7c-bebb-4834-9016-2e2cffcb7f5d-kube-api-access-mrlrg\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:45.547251 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.547220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8gk5\" (UniqueName: \"kubernetes.io/projected/0225b639-580d-4e3e-bb69-5faf45bf54b5-kube-api-access-q8gk5\") pod \"test-curl-7201-140378772937728\" (UID: \"0225b639-580d-4e3e-bb69-5faf45bf54b5\") " pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:45.648241 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.648176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8gk5\" (UniqueName: \"kubernetes.io/projected/0225b639-580d-4e3e-bb69-5faf45bf54b5-kube-api-access-q8gk5\") pod \"test-curl-7201-140378772937728\" (UID: \"0225b639-580d-4e3e-bb69-5faf45bf54b5\") " pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:45.659090 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.659065    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8gk5\" (UniqueName: \"kubernetes.io/projected/0225b639-580d-4e3e-bb69-5faf45bf54b5-kube-api-access-q8gk5\") pod \"test-curl-7201-140378772937728\" (UID: \"0225b639-580d-4e3e-bb69-5faf45bf54b5\") " pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:45.675836 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.675815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:45.808589 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:45.808564    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:45.810211 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:53:45.810184    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0225b639_580d_4e3e_bb69_5faf45bf54b5.slice/crio-581a7fc57ac12c05e3291a4caf41d8956938f05e1dfa3eae8bf94acf23436360 WatchSource:0}: Error finding container 581a7fc57ac12c05e3291a4caf41d8956938f05e1dfa3eae8bf94acf23436360: Status 404 returned error can't find the container with id 581a7fc57ac12c05e3291a4caf41d8956938f05e1dfa3eae8bf94acf23436360
Jul 28 14:53:46.029793 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:46.029768    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:46.029952 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:46.029768    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c3ce2e36441a5ef424c4431fc3fcbebe3a42d1286b5f3a8021611a087197683"
Jul 28 14:53:46.031119 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:46.031091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772937728" event={"ID":"0225b639-580d-4e3e-bb69-5faf45bf54b5","Type":"ContainerStarted","Data":"581a7fc57ac12c05e3291a4caf41d8956938f05e1dfa3eae8bf94acf23436360"}
Jul 28 14:53:46.041256 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:53:46.041225    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7201-140378772936720\" is forbidden: User \"system:node:ip-10-0-143-3.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-3.ec2.internal' and this object" podUID="d4c81d7c-bebb-4834-9016-2e2cffcb7f5d" pod="opendatahub/test-curl-7201-140378772936720"
Jul 28 14:53:46.342800 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:46.342727    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4c81d7c-bebb-4834-9016-2e2cffcb7f5d" path="/var/lib/kubelet/pods/d4c81d7c-bebb-4834-9016-2e2cffcb7f5d/volumes"
Jul 28 14:53:47.037853 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.037818    2579 generic.go:356] "Generic (PLEG): container finished" podID="0225b639-580d-4e3e-bb69-5faf45bf54b5" containerID="a300d0e273e0a01338a2040719d550f3ec5303da6b0f0190d880a5d64f7ec080" exitCode=0
Jul 28 14:53:47.038083 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.037905    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772937728" event={"ID":"0225b639-580d-4e3e-bb69-5faf45bf54b5","Type":"ContainerDied","Data":"a300d0e273e0a01338a2040719d550f3ec5303da6b0f0190d880a5d64f7ec080"}
Jul 28 14:53:47.537971 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.537929    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526" podUID="28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" containerName="main" containerID="cri-o://fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7" gracePeriod=45
Jul 28 14:53:47.794337 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.794283    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:53:47.970682 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.970654    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-dshm\") pod \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") "
Jul 28 14:53:47.970855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.970692    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-home\") pod \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") "
Jul 28 14:53:47.970855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.970725    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tmp-dir\") pod \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") "
Jul 28 14:53:47.970855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.970764    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tls-certs\") pod \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") "
Jul 28 14:53:47.970855 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.970812    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86qgg\" (UniqueName: \"kubernetes.io/projected/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-kube-api-access-86qgg\") pod \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") "
Jul 28 14:53:47.971129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.970870    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-model-cache\") pod \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\" (UID: \"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e\") "
Jul 28 14:53:47.971129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.970945    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-home" (OuterVolumeSpecName: "home") pod "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" (UID: "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:53:47.971129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.971050    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" (UID: "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:53:47.971280 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.971166    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" (UID: "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:53:47.971280 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.971213    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-home\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:47.971280 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.971227    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tmp-dir\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:47.972966 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.972942    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-dshm" (OuterVolumeSpecName: "dshm") pod "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" (UID: "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:53:47.973086 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.973033    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" (UID: "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:53:47.973235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:47.973211    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-kube-api-access-86qgg" (OuterVolumeSpecName: "kube-api-access-86qgg") pod "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" (UID: "28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e"). InnerVolumeSpecName "kube-api-access-86qgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:53:48.045077 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.044985    2579 generic.go:356] "Generic (PLEG): container finished" podID="28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" containerID="fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7" exitCode=0
Jul 28 14:53:48.045077 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.045050    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"
Jul 28 14:53:48.045077 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.045054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526" event={"ID":"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e","Type":"ContainerDied","Data":"fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7"}
Jul 28 14:53:48.045294 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.045099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526" event={"ID":"28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e","Type":"ContainerDied","Data":"42f038b52e95c1d1739d486377d40f51a7791f46fc47dc5a847343d970860e08"}
Jul 28 14:53:48.045294 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.045116    2579 scope.go:117] "RemoveContainer" containerID="fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7"
Jul 28 14:53:48.055044 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.055002    2579 scope.go:117] "RemoveContainer" containerID="fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7"
Jul 28 14:53:48.055285 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:53:48.055267    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7\": container with ID starting with fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7 not found: ID does not exist" containerID="fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7"
Jul 28 14:53:48.055330 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.055294    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7"} err="failed to get container status \"fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7\": rpc error: code = NotFound desc = could not find container \"fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7\": container with ID starting with fa314768a83b3030b4db0eec104f53aa1d9b292892c55cb05fbc13cd06d721d7 not found: ID does not exist"
Jul 28 14:53:48.069901 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.069877    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"]
Jul 28 14:53:48.071878 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.071857    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-tls-certs\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:48.071961 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.071882    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86qgg\" (UniqueName: \"kubernetes.io/projected/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-kube-api-access-86qgg\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:48.071961 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.071897    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-model-cache\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:48.071961 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.071909    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e-dshm\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:48.076418 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.076247    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-baa216/rate-test-model-baa216-kserve-64fc6cd95-7h526"]
Jul 28 14:53:48.169115 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.169087    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:48.191895 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.191869    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7201-140378772937728_0225b639-580d-4e3e-bb69-5faf45bf54b5/test-curl-7201-140378772937728/0.log"
Jul 28 14:53:48.222494 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.222465    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:48.226998 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.226973    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:48.273789 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.273769    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8gk5\" (UniqueName: \"kubernetes.io/projected/0225b639-580d-4e3e-bb69-5faf45bf54b5-kube-api-access-q8gk5\") pod \"0225b639-580d-4e3e-bb69-5faf45bf54b5\" (UID: \"0225b639-580d-4e3e-bb69-5faf45bf54b5\") "
Jul 28 14:53:48.275943 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.275922    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0225b639-580d-4e3e-bb69-5faf45bf54b5-kube-api-access-q8gk5" (OuterVolumeSpecName: "kube-api-access-q8gk5") pod "0225b639-580d-4e3e-bb69-5faf45bf54b5" (UID: "0225b639-580d-4e3e-bb69-5faf45bf54b5"). InnerVolumeSpecName "kube-api-access-q8gk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:53:48.342211 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.342142    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0225b639-580d-4e3e-bb69-5faf45bf54b5" path="/var/lib/kubelet/pods/0225b639-580d-4e3e-bb69-5faf45bf54b5/volumes"
Jul 28 14:53:48.342676 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.342651    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" path="/var/lib/kubelet/pods/28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e/volumes"
Jul 28 14:53:48.352253 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.352232    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:48.352586 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.352572    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0225b639-580d-4e3e-bb69-5faf45bf54b5" containerName="test-curl-7201-140378772937728"
Jul 28 14:53:48.352645 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.352587    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0225b639-580d-4e3e-bb69-5faf45bf54b5" containerName="test-curl-7201-140378772937728"
Jul 28 14:53:48.352645 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.352618    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" containerName="main"
Jul 28 14:53:48.352645 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.352623    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" containerName="main"
Jul 28 14:53:48.352737 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.352679    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ca0dd1-14e6-4d63-a48f-c50b5d1cab4e" containerName="main"
Jul 28 14:53:48.352737 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.352688    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0225b639-580d-4e3e-bb69-5faf45bf54b5" containerName="test-curl-7201-140378772937728"
Jul 28 14:53:48.355888 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.355874    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:48.362940 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.362921    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:48.374380 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.374355    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6znn\" (UniqueName: \"kubernetes.io/projected/5550b817-c7d9-43aa-ae6e-d4be9fe481b6-kube-api-access-t6znn\") pod \"test-curl-7201-140378772937728\" (UID: \"5550b817-c7d9-43aa-ae6e-d4be9fe481b6\") " pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:48.374497 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.374431    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8gk5\" (UniqueName: \"kubernetes.io/projected/0225b639-580d-4e3e-bb69-5faf45bf54b5-kube-api-access-q8gk5\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:48.475530 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.475490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6znn\" (UniqueName: \"kubernetes.io/projected/5550b817-c7d9-43aa-ae6e-d4be9fe481b6-kube-api-access-t6znn\") pod \"test-curl-7201-140378772937728\" (UID: \"5550b817-c7d9-43aa-ae6e-d4be9fe481b6\") " pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:48.485645 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.485621    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6znn\" (UniqueName: \"kubernetes.io/projected/5550b817-c7d9-43aa-ae6e-d4be9fe481b6-kube-api-access-t6znn\") pod \"test-curl-7201-140378772937728\" (UID: \"5550b817-c7d9-43aa-ae6e-d4be9fe481b6\") " pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:48.665235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.665147    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:48.798572 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:48.798545    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:48.799985 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:53:48.799961    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5550b817_c7d9_43aa_ae6e_d4be9fe481b6.slice/crio-fce0f3a86caa0cb96a121e456584d9abc7b815f34b58fa8a6d11d77734abcb09 WatchSource:0}: Error finding container fce0f3a86caa0cb96a121e456584d9abc7b815f34b58fa8a6d11d77734abcb09: Status 404 returned error can't find the container with id fce0f3a86caa0cb96a121e456584d9abc7b815f34b58fa8a6d11d77734abcb09
Jul 28 14:53:49.051998 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:49.051968    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:49.052201 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:49.052056    2579 scope.go:117] "RemoveContainer" containerID="a300d0e273e0a01338a2040719d550f3ec5303da6b0f0190d880a5d64f7ec080"
Jul 28 14:53:49.053541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:49.053500    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772937728" event={"ID":"5550b817-c7d9-43aa-ae6e-d4be9fe481b6","Type":"ContainerStarted","Data":"fce0f3a86caa0cb96a121e456584d9abc7b815f34b58fa8a6d11d77734abcb09"}
Jul 28 14:53:50.061556 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:50.061525    2579 generic.go:356] "Generic (PLEG): container finished" podID="5550b817-c7d9-43aa-ae6e-d4be9fe481b6" containerID="8af10d62f6ff69efa49685dc48a82bc9b8f3fc2067a8457785e7c9118c2826cb" exitCode=0
Jul 28 14:53:50.062008 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:50.061611    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7201-140378772937728" event={"ID":"5550b817-c7d9-43aa-ae6e-d4be9fe481b6","Type":"ContainerDied","Data":"8af10d62f6ff69efa49685dc48a82bc9b8f3fc2067a8457785e7c9118c2826cb"}
Jul 28 14:53:51.210035 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:51.209989    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:51.232046 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:51.231999    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7201-140378772937728_5550b817-c7d9-43aa-ae6e-d4be9fe481b6/test-curl-7201-140378772937728/0.log"
Jul 28 14:53:51.258578 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:51.258538    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:51.262889 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:51.262859    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7201-140378772937728"]
Jul 28 14:53:51.295380 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:51.295346    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6znn\" (UniqueName: \"kubernetes.io/projected/5550b817-c7d9-43aa-ae6e-d4be9fe481b6-kube-api-access-t6znn\") pod \"5550b817-c7d9-43aa-ae6e-d4be9fe481b6\" (UID: \"5550b817-c7d9-43aa-ae6e-d4be9fe481b6\") "
Jul 28 14:53:51.297540 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:51.297520    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5550b817-c7d9-43aa-ae6e-d4be9fe481b6-kube-api-access-t6znn" (OuterVolumeSpecName: "kube-api-access-t6znn") pod "5550b817-c7d9-43aa-ae6e-d4be9fe481b6" (UID: "5550b817-c7d9-43aa-ae6e-d4be9fe481b6"). InnerVolumeSpecName "kube-api-access-t6znn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:53:51.396317 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:51.396227    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6znn\" (UniqueName: \"kubernetes.io/projected/5550b817-c7d9-43aa-ae6e-d4be9fe481b6-kube-api-access-t6znn\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:53:52.074609 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:52.074582    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:52.074764 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:52.074624    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fce0f3a86caa0cb96a121e456584d9abc7b815f34b58fa8a6d11d77734abcb09"
Jul 28 14:53:52.084661 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:53:52.084637    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7201-140378772937728\" is forbidden: User \"system:node:ip-10-0-143-3.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-3.ec2.internal' and this object" podUID="5550b817-c7d9-43aa-ae6e-d4be9fe481b6" pod="opendatahub/test-curl-7201-140378772937728"
Jul 28 14:53:52.341801 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:52.341726    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5550b817-c7d9-43aa-ae6e-d4be9fe481b6" path="/var/lib/kubelet/pods/5550b817-c7d9-43aa-ae6e-d4be9fe481b6/volumes"
Jul 28 14:53:55.789502 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.789462    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"]
Jul 28 14:53:55.789984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.789972    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5550b817-c7d9-43aa-ae6e-d4be9fe481b6" containerName="test-curl-7201-140378772937728"
Jul 28 14:53:55.790083 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.789992    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5550b817-c7d9-43aa-ae6e-d4be9fe481b6" containerName="test-curl-7201-140378772937728"
Jul 28 14:53:55.790258 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.790240    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5550b817-c7d9-43aa-ae6e-d4be9fe481b6" containerName="test-curl-7201-140378772937728"
Jul 28 14:53:55.793411 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.793392    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:55.795836 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.795811    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-18ece0-serving-cert\""
Jul 28 14:53:55.795923 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.795853    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hsfxf\""
Jul 28 14:53:55.795923 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.795861    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 14:53:55.796686 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.796667    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 14:53:55.806061 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.806037    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"]
Jul 28 14:53:55.834445 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.834419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8p8r\" (UniqueName: \"kubernetes.io/projected/aceb6ade-22e2-48e5-8d57-f998a611c0c2-kube-api-access-f8p8r\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:55.834568 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.834461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aceb6ade-22e2-48e5-8d57-f998a611c0c2-maas-api-tls\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:55.834568 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.834558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aceb6ade-22e2-48e5-8d57-f998a611c0c2-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:55.935362 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.935330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8p8r\" (UniqueName: \"kubernetes.io/projected/aceb6ade-22e2-48e5-8d57-f998a611c0c2-kube-api-access-f8p8r\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:55.935490 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.935379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aceb6ade-22e2-48e5-8d57-f998a611c0c2-maas-api-tls\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:55.935490 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.935420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aceb6ade-22e2-48e5-8d57-f998a611c0c2-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:55.935567 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:53:55.935521    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-serving-cert: secret "maas-api-e2e-ipp-a-18ece0-serving-cert" not found
Jul 28 14:53:55.935602 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:53:55.935590    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aceb6ade-22e2-48e5-8d57-f998a611c0c2-maas-api-tls podName:aceb6ade-22e2-48e5-8d57-f998a611c0c2 nodeName:}" failed. No retries permitted until 2026-07-28 14:53:56.435569999 +0000 UTC m=+3252.562960670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aceb6ade-22e2-48e5-8d57-f998a611c0c2-maas-api-tls") pod "maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h" (UID: "aceb6ade-22e2-48e5-8d57-f998a611c0c2") : secret "maas-api-e2e-ipp-a-18ece0-serving-cert" not found
Jul 28 14:53:55.936087 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.936071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aceb6ade-22e2-48e5-8d57-f998a611c0c2-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:55.944402 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:55.944384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8p8r\" (UniqueName: \"kubernetes.io/projected/aceb6ade-22e2-48e5-8d57-f998a611c0c2-kube-api-access-f8p8r\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:56.438616 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:56.438572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aceb6ade-22e2-48e5-8d57-f998a611c0c2-maas-api-tls\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:56.441293 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:56.441268    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aceb6ade-22e2-48e5-8d57-f998a611c0c2-maas-api-tls\") pod \"maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:56.704560 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:56.704457    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:56.859002 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:56.858977    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"]
Jul 28 14:53:56.860924 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:53:56.860898    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaceb6ade_22e2_48e5_8d57_f998a611c0c2.slice/crio-04aa56116dbe250d9146dc14d48b15c834aa12a5258d61fdd1fc8d0f393c635f WatchSource:0}: Error finding container 04aa56116dbe250d9146dc14d48b15c834aa12a5258d61fdd1fc8d0f393c635f: Status 404 returned error can't find the container with id 04aa56116dbe250d9146dc14d48b15c834aa12a5258d61fdd1fc8d0f393c635f
Jul 28 14:53:57.102362 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:57.102327    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h" event={"ID":"aceb6ade-22e2-48e5-8d57-f998a611c0c2","Type":"ContainerStarted","Data":"04aa56116dbe250d9146dc14d48b15c834aa12a5258d61fdd1fc8d0f393c635f"}
Jul 28 14:53:58.109747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:58.109709    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h" event={"ID":"aceb6ade-22e2-48e5-8d57-f998a611c0c2","Type":"ContainerStarted","Data":"ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2"}
Jul 28 14:53:58.110263 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:58.109796    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:53:58.130841 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:53:58.130790    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h" podStartSLOduration=2.553501977 podStartE2EDuration="3.130776866s" totalImagesPullingTime="577.274889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:53:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:53:58.128897808 +0000 UTC m=+3254.256288500" watchObservedRunningTime="2026-07-28 14:53:58.130776866 +0000 UTC m=+3254.258167557"
Jul 28 14:54:02.140373 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:02.140338    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:54:04.120796 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:04.120765    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:54:26.212277 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.212243    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"]
Jul 28 14:54:26.216470 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.216451    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.218537 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.218515    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-fd71df-serving-cert\""
Jul 28 14:54:26.226316 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.226292    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"]
Jul 28 14:54:26.327796 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.327755    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.327968 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.327818    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nzt2\" (UniqueName: \"kubernetes.io/projected/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-kube-api-access-6nzt2\") pod \"maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.327968 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.327903    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.429279 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.429245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.429447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.429309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nzt2\" (UniqueName: \"kubernetes.io/projected/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-kube-api-access-6nzt2\") pod \"maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.429447 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.429348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.429995 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.429978    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.431832 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.431811    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.438707 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.438673    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nzt2\" (UniqueName: \"kubernetes.io/projected/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-kube-api-access-6nzt2\") pod \"maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.529201 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.529172    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:26.675218 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:26.675194    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"]
Jul 28 14:54:26.676860 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:54:26.676835    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7bcd0e8_ceba_4dfa_913e_c2bbd7331b1c.slice/crio-eb15c3811a0ebba05ff42f2a3f997400bf22b9029602dc47e0fe6b4f7326406e WatchSource:0}: Error finding container eb15c3811a0ebba05ff42f2a3f997400bf22b9029602dc47e0fe6b4f7326406e: Status 404 returned error can't find the container with id eb15c3811a0ebba05ff42f2a3f997400bf22b9029602dc47e0fe6b4f7326406e
Jul 28 14:54:27.271180 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:27.271142    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8" event={"ID":"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c","Type":"ContainerStarted","Data":"8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a"}
Jul 28 14:54:27.271577 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:27.271200    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8" event={"ID":"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c","Type":"ContainerStarted","Data":"eb15c3811a0ebba05ff42f2a3f997400bf22b9029602dc47e0fe6b4f7326406e"}
Jul 28 14:54:27.271577 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:27.271303    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:27.289807 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:27.289749    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8" podStartSLOduration=0.862368391 podStartE2EDuration="1.28973426s" totalImagesPullingTime="427.365869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:54:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:54:27.288321271 +0000 UTC m=+3283.415711962" watchObservedRunningTime="2026-07-28 14:54:27.28973426 +0000 UTC m=+3283.417124953"
Jul 28 14:54:33.281871 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:33.281840    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:54:34.830454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:34.830419    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:54:44.595365 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:44.595338    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:54:44.606964 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:44.606943    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:54:58.038500 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.038464    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"]
Jul 28 14:54:58.042063 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.042045    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.044570 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.044547    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-18ece0\"/\"kube-root-ca.crt\""
Jul 28 14:54:58.045374 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.045352    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-18ece0\"/\"ipp-route-18ece0-kserve-self-signed-certs\""
Jul 28 14:54:58.045464 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.045450    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-18ece0\"/\"default-dockercfg-mj4z9\""
Jul 28 14:54:58.045517 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.045463    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-18ece0\"/\"openshift-service-ca.crt\""
Jul 28 14:54:58.052997 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.052969    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"]
Jul 28 14:54:58.111454 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.111424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-home\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.111622 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.111458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-dshm\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.111622 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.111481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-tmp-dir\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.111622 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.111608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0091b045-80fc-4a30-8f69-e3f84ab41d99-tls-certs\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.111752 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.111639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxxzp\" (UniqueName: \"kubernetes.io/projected/0091b045-80fc-4a30-8f69-e3f84ab41d99-kube-api-access-dxxzp\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.111752 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.111666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-model-cache\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.212465 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.212432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0091b045-80fc-4a30-8f69-e3f84ab41d99-tls-certs\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.212465 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.212465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxxzp\" (UniqueName: \"kubernetes.io/projected/0091b045-80fc-4a30-8f69-e3f84ab41d99-kube-api-access-dxxzp\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.212693 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.212486    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-model-cache\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.212693 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.212523    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-home\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.212693 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.212647    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-dshm\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.212861 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.212701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-tmp-dir\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.212919 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.212891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-model-cache\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.212977 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.212951    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-home\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.213086 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.213064    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-tmp-dir\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.214986 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.214964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-dshm\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.215390 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.215368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0091b045-80fc-4a30-8f69-e3f84ab41d99-tls-certs\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.221700 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.221681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxxzp\" (UniqueName: \"kubernetes.io/projected/0091b045-80fc-4a30-8f69-e3f84ab41d99-kube-api-access-dxxzp\") pod \"ipp-route-18ece0-kserve-79964fbc5f-kzdv4\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") " pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.352823 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.352744    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:54:58.492430 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:58.492396    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"]
Jul 28 14:54:58.494053 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:54:58.494002    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0091b045_80fc_4a30_8f69_e3f84ab41d99.slice/crio-a1d726cd44dc728314cc9aa953e7038d24f812218dc75f2f3dc81ba215afa127 WatchSource:0}: Error finding container a1d726cd44dc728314cc9aa953e7038d24f812218dc75f2f3dc81ba215afa127: Status 404 returned error can't find the container with id a1d726cd44dc728314cc9aa953e7038d24f812218dc75f2f3dc81ba215afa127
Jul 28 14:54:59.450462 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:59.450428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4" event={"ID":"0091b045-80fc-4a30-8f69-e3f84ab41d99","Type":"ContainerStarted","Data":"c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173"}
Jul 28 14:54:59.450850 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:59.450469    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4" event={"ID":"0091b045-80fc-4a30-8f69-e3f84ab41d99","Type":"ContainerStarted","Data":"a1d726cd44dc728314cc9aa953e7038d24f812218dc75f2f3dc81ba215afa127"}
Jul 28 14:54:59.470581 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:54:59.470532    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4" podStartSLOduration=1.4705172260000001 podStartE2EDuration="1.470517226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:54:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:54:59.469122764 +0000 UTC m=+3315.596513456" watchObservedRunningTime="2026-07-28 14:54:59.470517226 +0000 UTC m=+3315.597907916"
Jul 28 14:55:02.342129 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:02.342102    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:55:08.353819 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:08.353785    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:55:08.354305 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:08.353833    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:55:08.366582 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:08.366554    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:55:08.515483 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:08.515452    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:55:32.140113 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:32.140081    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:55:48.927284 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:48.927251    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"]
Jul 28 14:55:48.931238 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:48.931208    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:48.933640 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:48.933615    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-694dae-serving-cert\""
Jul 28 14:55:48.942291 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:48.942263    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"]
Jul 28 14:55:48.985209 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:48.985174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a73a737-95b0-45cc-9819-cb6cd1096780-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:48.985390 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:48.985215    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a73a737-95b0-45cc-9819-cb6cd1096780-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:48.985390 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:48.985304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l7gn\" (UniqueName: \"kubernetes.io/projected/2a73a737-95b0-45cc-9819-cb6cd1096780-kube-api-access-9l7gn\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:49.086699 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:49.086659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a73a737-95b0-45cc-9819-cb6cd1096780-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:49.086884 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:49.086708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a73a737-95b0-45cc-9819-cb6cd1096780-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:49.086884 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:49.086759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7gn\" (UniqueName: \"kubernetes.io/projected/2a73a737-95b0-45cc-9819-cb6cd1096780-kube-api-access-9l7gn\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:49.086959 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:55:49.086897    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-serving-cert: secret "maas-api-e2e-ipp-cleanup-694dae-serving-cert" not found
Jul 28 14:55:49.087002 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:55:49.086990    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a73a737-95b0-45cc-9819-cb6cd1096780-maas-api-tls podName:2a73a737-95b0-45cc-9819-cb6cd1096780 nodeName:}" failed. No retries permitted until 2026-07-28 14:55:49.586968304 +0000 UTC m=+3365.714358977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a73a737-95b0-45cc-9819-cb6cd1096780-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7" (UID: "2a73a737-95b0-45cc-9819-cb6cd1096780") : secret "maas-api-e2e-ipp-cleanup-694dae-serving-cert" not found
Jul 28 14:55:49.087522 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:49.087506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a73a737-95b0-45cc-9819-cb6cd1096780-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:49.096676 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:49.096649    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l7gn\" (UniqueName: \"kubernetes.io/projected/2a73a737-95b0-45cc-9819-cb6cd1096780-kube-api-access-9l7gn\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:49.592400 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:49.592368    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a73a737-95b0-45cc-9819-cb6cd1096780-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:49.594939 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:49.594920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a73a737-95b0-45cc-9819-cb6cd1096780-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:49.844524 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:49.844423    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:49.986737 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:49.986693    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"]
Jul 28 14:55:49.988988 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:55:49.988959    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a73a737_95b0_45cc_9819_cb6cd1096780.slice/crio-82c4f01cb2574693cbd38da57a31a4555ae3bc3578f0c9ced324bf711c3d455e WatchSource:0}: Error finding container 82c4f01cb2574693cbd38da57a31a4555ae3bc3578f0c9ced324bf711c3d455e: Status 404 returned error can't find the container with id 82c4f01cb2574693cbd38da57a31a4555ae3bc3578f0c9ced324bf711c3d455e
Jul 28 14:55:50.750535 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:50.750431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7" event={"ID":"2a73a737-95b0-45cc-9819-cb6cd1096780","Type":"ContainerStarted","Data":"1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd"}
Jul 28 14:55:50.750535 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:50.750494    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7" event={"ID":"2a73a737-95b0-45cc-9819-cb6cd1096780","Type":"ContainerStarted","Data":"82c4f01cb2574693cbd38da57a31a4555ae3bc3578f0c9ced324bf711c3d455e"}
Jul 28 14:55:50.750535 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:50.750525    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:50.776333 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:50.776275    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7" podStartSLOduration=2.386005687 podStartE2EDuration="2.776260601s" totalImagesPullingTime="390.254914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:55:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:55:50.774443333 +0000 UTC m=+3366.901834040" watchObservedRunningTime="2026-07-28 14:55:50.776260601 +0000 UTC m=+3366.903651298"
Jul 28 14:55:56.762581 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:56.762549    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:55:57.064604 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:55:57.064510    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:56:03.456875 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:03.456833    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"]
Jul 28 14:56:03.460875 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:03.460851    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"
Jul 28 14:56:03.463185 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:03.463159    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:56:03.468495 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:03.468464    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"]
Jul 28 14:56:03.623134 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:03.623094    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20ae368e-d9d0-49b0-bf08-b69a0db2e0aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4\" (UID: \"20ae368e-d9d0-49b0-bf08-b69a0db2e0aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"
Jul 28 14:56:03.724436 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:03.724331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20ae368e-d9d0-49b0-bf08-b69a0db2e0aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4\" (UID: \"20ae368e-d9d0-49b0-bf08-b69a0db2e0aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"
Jul 28 14:56:03.725046 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:03.724995    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20ae368e-d9d0-49b0-bf08-b69a0db2e0aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4\" (UID: \"20ae368e-d9d0-49b0-bf08-b69a0db2e0aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"
Jul 28 14:56:03.773087 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:03.772984    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"
Jul 28 14:56:03.919041 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:03.918996    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"]
Jul 28 14:56:03.920751 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:56:03.920722    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ae368e_d9d0_49b0_bf08_b69a0db2e0aa.slice/crio-ccd79ab3eaa8f51d6c2304da037a4d7f16787654861e275925cd349eb5e8536c WatchSource:0}: Error finding container ccd79ab3eaa8f51d6c2304da037a4d7f16787654861e275925cd349eb5e8536c: Status 404 returned error can't find the container with id ccd79ab3eaa8f51d6c2304da037a4d7f16787654861e275925cd349eb5e8536c
Jul 28 14:56:04.839887 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:04.839783    2579 generic.go:356] "Generic (PLEG): container finished" podID="20ae368e-d9d0-49b0-bf08-b69a0db2e0aa" containerID="b68006ffeee3e545dd11ed85444db7a5b0c830b18d564226f2478d13b8ae51b1" exitCode=0
Jul 28 14:56:04.840327 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:04.839873    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4" event={"ID":"20ae368e-d9d0-49b0-bf08-b69a0db2e0aa","Type":"ContainerDied","Data":"b68006ffeee3e545dd11ed85444db7a5b0c830b18d564226f2478d13b8ae51b1"}
Jul 28 14:56:04.840327 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:04.839944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4" event={"ID":"20ae368e-d9d0-49b0-bf08-b69a0db2e0aa","Type":"ContainerStarted","Data":"ccd79ab3eaa8f51d6c2304da037a4d7f16787654861e275925cd349eb5e8536c"}
Jul 28 14:56:05.983947 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:05.983923    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"
Jul 28 14:56:06.148096 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:06.147594    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20ae368e-d9d0-49b0-bf08-b69a0db2e0aa-maas-api-service-ca\") pod \"20ae368e-d9d0-49b0-bf08-b69a0db2e0aa\" (UID: \"20ae368e-d9d0-49b0-bf08-b69a0db2e0aa\") "
Jul 28 14:56:06.148672 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:06.148637    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ae368e-d9d0-49b0-bf08-b69a0db2e0aa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "20ae368e-d9d0-49b0-bf08-b69a0db2e0aa" (UID: "20ae368e-d9d0-49b0-bf08-b69a0db2e0aa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:56:06.249485 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:06.249443    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20ae368e-d9d0-49b0-bf08-b69a0db2e0aa-maas-api-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:06.853715 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:06.853681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4" event={"ID":"20ae368e-d9d0-49b0-bf08-b69a0db2e0aa","Type":"ContainerDied","Data":"ccd79ab3eaa8f51d6c2304da037a4d7f16787654861e275925cd349eb5e8536c"}
Jul 28 14:56:06.853715 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:06.853703    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"
Jul 28 14:56:06.853983 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:06.853731    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccd79ab3eaa8f51d6c2304da037a4d7f16787654861e275925cd349eb5e8536c"
Jul 28 14:56:11.565175 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.565142    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"]
Jul 28 14:56:11.565595 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.565388    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7" podUID="2a73a737-95b0-45cc-9819-cb6cd1096780" containerName="maas-api" containerID="cri-o://1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd" gracePeriod=30
Jul 28 14:56:11.826926 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.826851    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:56:11.869507 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.869476    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"]
Jul 28 14:56:11.877372 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.877342    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-694dae-mzcq4"]
Jul 28 14:56:11.884530 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.884504    2579 generic.go:356] "Generic (PLEG): container finished" podID="2a73a737-95b0-45cc-9819-cb6cd1096780" containerID="1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd" exitCode=0
Jul 28 14:56:11.884708 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.884551    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"
Jul 28 14:56:11.884708 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.884585    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7" event={"ID":"2a73a737-95b0-45cc-9819-cb6cd1096780","Type":"ContainerDied","Data":"1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd"}
Jul 28 14:56:11.884708 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.884652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7" event={"ID":"2a73a737-95b0-45cc-9819-cb6cd1096780","Type":"ContainerDied","Data":"82c4f01cb2574693cbd38da57a31a4555ae3bc3578f0c9ced324bf711c3d455e"}
Jul 28 14:56:11.884708 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.884683    2579 scope.go:117] "RemoveContainer" containerID="1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd"
Jul 28 14:56:11.894963 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.894944    2579 scope.go:117] "RemoveContainer" containerID="1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd"
Jul 28 14:56:11.895331 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:56:11.895296    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd\": container with ID starting with 1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd not found: ID does not exist" containerID="1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd"
Jul 28 14:56:11.895414 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.895340    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd"} err="failed to get container status \"1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd\": rpc error: code = NotFound desc = could not find container \"1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd\": container with ID starting with 1de896a7f99bdf4f4f313a357475873850e73d4b96ff3844eae5b46880c6dcbd not found: ID does not exist"
Jul 28 14:56:11.909707 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.909679    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a73a737-95b0-45cc-9819-cb6cd1096780-openshift-service-ca\") pod \"2a73a737-95b0-45cc-9819-cb6cd1096780\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") "
Jul 28 14:56:11.909864 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.909735    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l7gn\" (UniqueName: \"kubernetes.io/projected/2a73a737-95b0-45cc-9819-cb6cd1096780-kube-api-access-9l7gn\") pod \"2a73a737-95b0-45cc-9819-cb6cd1096780\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") "
Jul 28 14:56:11.909864 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.909822    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a73a737-95b0-45cc-9819-cb6cd1096780-maas-api-tls\") pod \"2a73a737-95b0-45cc-9819-cb6cd1096780\" (UID: \"2a73a737-95b0-45cc-9819-cb6cd1096780\") "
Jul 28 14:56:11.910153 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.910126    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a73a737-95b0-45cc-9819-cb6cd1096780-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a73a737-95b0-45cc-9819-cb6cd1096780" (UID: "2a73a737-95b0-45cc-9819-cb6cd1096780"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:56:11.912137 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.912110    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a73a737-95b0-45cc-9819-cb6cd1096780-kube-api-access-9l7gn" (OuterVolumeSpecName: "kube-api-access-9l7gn") pod "2a73a737-95b0-45cc-9819-cb6cd1096780" (UID: "2a73a737-95b0-45cc-9819-cb6cd1096780"). InnerVolumeSpecName "kube-api-access-9l7gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:56:11.912337 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:11.912312    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a73a737-95b0-45cc-9819-cb6cd1096780-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a73a737-95b0-45cc-9819-cb6cd1096780" (UID: "2a73a737-95b0-45cc-9819-cb6cd1096780"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:56:12.011407 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:12.011355    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l7gn\" (UniqueName: \"kubernetes.io/projected/2a73a737-95b0-45cc-9819-cb6cd1096780-kube-api-access-9l7gn\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:12.011407 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:12.011402    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a73a737-95b0-45cc-9819-cb6cd1096780-maas-api-tls\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:12.011407 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:12.011411    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a73a737-95b0-45cc-9819-cb6cd1096780-openshift-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:12.211812 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:12.211777    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"]
Jul 28 14:56:12.219616 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:12.219587    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7"]
Jul 28 14:56:12.342169 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:12.342130    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20ae368e-d9d0-49b0-bf08-b69a0db2e0aa" path="/var/lib/kubelet/pods/20ae368e-d9d0-49b0-bf08-b69a0db2e0aa/volumes"
Jul 28 14:56:12.342584 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:12.342567    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a73a737-95b0-45cc-9819-cb6cd1096780" path="/var/lib/kubelet/pods/2a73a737-95b0-45cc-9819-cb6cd1096780/volumes"
Jul 28 14:56:14.758214 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:14.758164    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-694dae-7765955b84-wlzx7" podUID="2a73a737-95b0-45cc-9819-cb6cd1096780" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.76:8443/health\": context deadline exceeded"
Jul 28 14:56:16.176040 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:16.175980    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:56:23.731241 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.731209    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"]
Jul 28 14:56:23.731642 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.731620    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a73a737-95b0-45cc-9819-cb6cd1096780" containerName="maas-api"
Jul 28 14:56:23.731642 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.731630    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a73a737-95b0-45cc-9819-cb6cd1096780" containerName="maas-api"
Jul 28 14:56:23.731712 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.731644    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20ae368e-d9d0-49b0-bf08-b69a0db2e0aa" containerName="revoke-keys"
Jul 28 14:56:23.731712 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.731650    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="20ae368e-d9d0-49b0-bf08-b69a0db2e0aa" containerName="revoke-keys"
Jul 28 14:56:23.731712 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.731708    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a73a737-95b0-45cc-9819-cb6cd1096780" containerName="maas-api"
Jul 28 14:56:23.731811 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.731718    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="20ae368e-d9d0-49b0-bf08-b69a0db2e0aa" containerName="revoke-keys"
Jul 28 14:56:23.736383 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.736365    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"
Jul 28 14:56:23.738697 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.738670    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:56:23.743997 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.743728    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"]
Jul 28 14:56:23.822565 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.822530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4\" (UID: \"4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"
Jul 28 14:56:23.923325 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.923280    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4\" (UID: \"4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"
Jul 28 14:56:23.923978 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:23.923952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4\" (UID: \"4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"
Jul 28 14:56:24.047647 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:24.047612    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"
Jul 28 14:56:24.189391 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:24.189362    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"]
Jul 28 14:56:24.190964 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:56:24.190933    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e78b9a0_87ee_403f_a6c7_b707bb1c7f7d.slice/crio-83c68c5f7a47bd19c61aee43a479c449130f809729cfd7d69a6b1af9954a7f8d WatchSource:0}: Error finding container 83c68c5f7a47bd19c61aee43a479c449130f809729cfd7d69a6b1af9954a7f8d: Status 404 returned error can't find the container with id 83c68c5f7a47bd19c61aee43a479c449130f809729cfd7d69a6b1af9954a7f8d
Jul 28 14:56:24.963122 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:24.963085    2579 generic.go:356] "Generic (PLEG): container finished" podID="4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d" containerID="2654bf463d90a0b7efb6eb6d58e677f3b0643287bec2790deb31363392f5b394" exitCode=0
Jul 28 14:56:24.963522 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:24.963176    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4" event={"ID":"4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d","Type":"ContainerDied","Data":"2654bf463d90a0b7efb6eb6d58e677f3b0643287bec2790deb31363392f5b394"}
Jul 28 14:56:24.963522 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:24.963228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4" event={"ID":"4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d","Type":"ContainerStarted","Data":"83c68c5f7a47bd19c61aee43a479c449130f809729cfd7d69a6b1af9954a7f8d"}
Jul 28 14:56:26.103590 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:26.103564    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"
Jul 28 14:56:26.143445 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:26.143420    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d-maas-api-service-ca\") pod \"4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d\" (UID: \"4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d\") "
Jul 28 14:56:26.143780 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:26.143761    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d" (UID: "4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:56:26.244591 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:26.244502    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d-maas-api-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:26.977344 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:26.977312    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4" event={"ID":"4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d","Type":"ContainerDied","Data":"83c68c5f7a47bd19c61aee43a479c449130f809729cfd7d69a6b1af9954a7f8d"}
Jul 28 14:56:26.977344 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:26.977343    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"
Jul 28 14:56:26.977560 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:26.977359    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83c68c5f7a47bd19c61aee43a479c449130f809729cfd7d69a6b1af9954a7f8d"
Jul 28 14:56:38.496166 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.496085    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:56:38.579219 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.579186    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"]
Jul 28 14:56:38.579465 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.579434    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h" podUID="aceb6ade-22e2-48e5-8d57-f998a611c0c2" containerName="maas-api" containerID="cri-o://ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2" gracePeriod=30
Jul 28 14:56:38.837035 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.836991    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:56:38.961759 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.961725    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8p8r\" (UniqueName: \"kubernetes.io/projected/aceb6ade-22e2-48e5-8d57-f998a611c0c2-kube-api-access-f8p8r\") pod \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") "
Jul 28 14:56:38.961958 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.961789    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aceb6ade-22e2-48e5-8d57-f998a611c0c2-maas-api-tls\") pod \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") "
Jul 28 14:56:38.961958 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.961830    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aceb6ade-22e2-48e5-8d57-f998a611c0c2-openshift-service-ca\") pod \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\" (UID: \"aceb6ade-22e2-48e5-8d57-f998a611c0c2\") "
Jul 28 14:56:38.962279 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.962254    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aceb6ade-22e2-48e5-8d57-f998a611c0c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aceb6ade-22e2-48e5-8d57-f998a611c0c2" (UID: "aceb6ade-22e2-48e5-8d57-f998a611c0c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:56:38.964109 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.964083    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aceb6ade-22e2-48e5-8d57-f998a611c0c2-kube-api-access-f8p8r" (OuterVolumeSpecName: "kube-api-access-f8p8r") pod "aceb6ade-22e2-48e5-8d57-f998a611c0c2" (UID: "aceb6ade-22e2-48e5-8d57-f998a611c0c2"). InnerVolumeSpecName "kube-api-access-f8p8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:56:38.964209 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.964195    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aceb6ade-22e2-48e5-8d57-f998a611c0c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aceb6ade-22e2-48e5-8d57-f998a611c0c2" (UID: "aceb6ade-22e2-48e5-8d57-f998a611c0c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:56:38.975549 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.975524    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"]
Jul 28 14:56:38.978483 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:38.978461    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-18ece0-5r6x4"]
Jul 28 14:56:39.052785 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.052741    2579 generic.go:356] "Generic (PLEG): container finished" podID="aceb6ade-22e2-48e5-8d57-f998a611c0c2" containerID="ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2" exitCode=0
Jul 28 14:56:39.052785 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.052786    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"
Jul 28 14:56:39.053098 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.052820    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h" event={"ID":"aceb6ade-22e2-48e5-8d57-f998a611c0c2","Type":"ContainerDied","Data":"ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2"}
Jul 28 14:56:39.053098 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.052876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h" event={"ID":"aceb6ade-22e2-48e5-8d57-f998a611c0c2","Type":"ContainerDied","Data":"04aa56116dbe250d9146dc14d48b15c834aa12a5258d61fdd1fc8d0f393c635f"}
Jul 28 14:56:39.053098 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.052897    2579 scope.go:117] "RemoveContainer" containerID="ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2"
Jul 28 14:56:39.063320 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.063299    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8p8r\" (UniqueName: \"kubernetes.io/projected/aceb6ade-22e2-48e5-8d57-f998a611c0c2-kube-api-access-f8p8r\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:39.063320 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.063320    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aceb6ade-22e2-48e5-8d57-f998a611c0c2-maas-api-tls\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:39.063466 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.063330    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aceb6ade-22e2-48e5-8d57-f998a611c0c2-openshift-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:39.063597 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.063577    2579 scope.go:117] "RemoveContainer" containerID="ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2"
Jul 28 14:56:39.063950 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:56:39.063931    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2\": container with ID starting with ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2 not found: ID does not exist" containerID="ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2"
Jul 28 14:56:39.064006 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.063959    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2"} err="failed to get container status \"ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2\": rpc error: code = NotFound desc = could not find container \"ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2\": container with ID starting with ce5f27ac988f5d2303c4f8714ccd2f626dae631a55ace191a7bf2a39f7a22bc2 not found: ID does not exist"
Jul 28 14:56:39.077616 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.077246    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"]
Jul 28 14:56:39.081917 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:39.081889    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-18ece0-7ffbb8598b-zpn5h"]
Jul 28 14:56:40.342133 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:40.342101    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d" path="/var/lib/kubelet/pods/4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d/volumes"
Jul 28 14:56:40.342497 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:40.342468    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aceb6ade-22e2-48e5-8d57-f998a611c0c2" path="/var/lib/kubelet/pods/aceb6ade-22e2-48e5-8d57-f998a611c0c2/volumes"
Jul 28 14:56:44.361331 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:44.361294    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"]
Jul 28 14:56:49.232199 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:49.232167    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:56:59.381107 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.381062    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4" podUID="0091b045-80fc-4a30-8f69-e3f84ab41d99" containerName="main" containerID="cri-o://c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173" gracePeriod=45
Jul 28 14:56:59.650714 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.650688    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:56:59.753726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.753691    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-model-cache\") pod \"0091b045-80fc-4a30-8f69-e3f84ab41d99\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") "
Jul 28 14:56:59.753726 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.753723    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-dshm\") pod \"0091b045-80fc-4a30-8f69-e3f84ab41d99\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") "
Jul 28 14:56:59.753971 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.753767    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-home\") pod \"0091b045-80fc-4a30-8f69-e3f84ab41d99\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") "
Jul 28 14:56:59.753971 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.753809    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxxzp\" (UniqueName: \"kubernetes.io/projected/0091b045-80fc-4a30-8f69-e3f84ab41d99-kube-api-access-dxxzp\") pod \"0091b045-80fc-4a30-8f69-e3f84ab41d99\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") "
Jul 28 14:56:59.753971 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.753853    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-tmp-dir\") pod \"0091b045-80fc-4a30-8f69-e3f84ab41d99\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") "
Jul 28 14:56:59.754162 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.754003    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0091b045-80fc-4a30-8f69-e3f84ab41d99-tls-certs\") pod \"0091b045-80fc-4a30-8f69-e3f84ab41d99\" (UID: \"0091b045-80fc-4a30-8f69-e3f84ab41d99\") "
Jul 28 14:56:59.754162 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.754065    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-model-cache" (OuterVolumeSpecName: "model-cache") pod "0091b045-80fc-4a30-8f69-e3f84ab41d99" (UID: "0091b045-80fc-4a30-8f69-e3f84ab41d99"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:56:59.754162 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.754135    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-home" (OuterVolumeSpecName: "home") pod "0091b045-80fc-4a30-8f69-e3f84ab41d99" (UID: "0091b045-80fc-4a30-8f69-e3f84ab41d99"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:56:59.754162 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.754145    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0091b045-80fc-4a30-8f69-e3f84ab41d99" (UID: "0091b045-80fc-4a30-8f69-e3f84ab41d99"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:56:59.754371 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.754351    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-home\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:59.754426 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.754369    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-tmp-dir\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:59.754426 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.754383    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-model-cache\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:59.756230 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.756204    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-dshm" (OuterVolumeSpecName: "dshm") pod "0091b045-80fc-4a30-8f69-e3f84ab41d99" (UID: "0091b045-80fc-4a30-8f69-e3f84ab41d99"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:56:59.756335 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.756267    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0091b045-80fc-4a30-8f69-e3f84ab41d99-kube-api-access-dxxzp" (OuterVolumeSpecName: "kube-api-access-dxxzp") pod "0091b045-80fc-4a30-8f69-e3f84ab41d99" (UID: "0091b045-80fc-4a30-8f69-e3f84ab41d99"). InnerVolumeSpecName "kube-api-access-dxxzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:56:59.756503 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.756484    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0091b045-80fc-4a30-8f69-e3f84ab41d99-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0091b045-80fc-4a30-8f69-e3f84ab41d99" (UID: "0091b045-80fc-4a30-8f69-e3f84ab41d99"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:56:59.855231 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.855194    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0091b045-80fc-4a30-8f69-e3f84ab41d99-tls-certs\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:59.855231 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.855223    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0091b045-80fc-4a30-8f69-e3f84ab41d99-dshm\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:56:59.855231 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:56:59.855233    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxxzp\" (UniqueName: \"kubernetes.io/projected/0091b045-80fc-4a30-8f69-e3f84ab41d99-kube-api-access-dxxzp\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:57:00.182345 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.182308    2579 generic.go:356] "Generic (PLEG): container finished" podID="0091b045-80fc-4a30-8f69-e3f84ab41d99" containerID="c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173" exitCode=0
Jul 28 14:57:00.182559 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.182358    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"
Jul 28 14:57:00.182559 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.182374    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4" event={"ID":"0091b045-80fc-4a30-8f69-e3f84ab41d99","Type":"ContainerDied","Data":"c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173"}
Jul 28 14:57:00.182559 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.182415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4" event={"ID":"0091b045-80fc-4a30-8f69-e3f84ab41d99","Type":"ContainerDied","Data":"a1d726cd44dc728314cc9aa953e7038d24f812218dc75f2f3dc81ba215afa127"}
Jul 28 14:57:00.182559 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.182436    2579 scope.go:117] "RemoveContainer" containerID="c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173"
Jul 28 14:57:00.192984 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.192962    2579 scope.go:117] "RemoveContainer" containerID="c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173"
Jul 28 14:57:00.193291 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:57:00.193273    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173\": container with ID starting with c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173 not found: ID does not exist" containerID="c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173"
Jul 28 14:57:00.193347 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.193301    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173"} err="failed to get container status \"c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173\": rpc error: code = NotFound desc = could not find container \"c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173\": container with ID starting with c167f68d1d068c3ef3233c01f7e834e400586e65ae5d9062ae3c0af50fc35173 not found: ID does not exist"
Jul 28 14:57:00.205534 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.205503    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"]
Jul 28 14:57:00.209319 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.209290    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-18ece0/ipp-route-18ece0-kserve-79964fbc5f-kzdv4"]
Jul 28 14:57:00.342163 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:00.342134    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0091b045-80fc-4a30-8f69-e3f84ab41d99" path="/var/lib/kubelet/pods/0091b045-80fc-4a30-8f69-e3f84ab41d99/volumes"
Jul 28 14:57:30.933151 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933120    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"]
Jul 28 14:57:30.933631 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933511    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aceb6ade-22e2-48e5-8d57-f998a611c0c2" containerName="maas-api"
Jul 28 14:57:30.933631 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933522    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aceb6ade-22e2-48e5-8d57-f998a611c0c2" containerName="maas-api"
Jul 28 14:57:30.933631 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933531    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0091b045-80fc-4a30-8f69-e3f84ab41d99" containerName="main"
Jul 28 14:57:30.933631 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933536    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0091b045-80fc-4a30-8f69-e3f84ab41d99" containerName="main"
Jul 28 14:57:30.933631 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933545    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d" containerName="revoke-keys"
Jul 28 14:57:30.933631 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933551    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d" containerName="revoke-keys"
Jul 28 14:57:30.933631 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933619    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0091b045-80fc-4a30-8f69-e3f84ab41d99" containerName="main"
Jul 28 14:57:30.933631 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933626    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e78b9a0-87ee-403f-a6c7-b707bb1c7f7d" containerName="revoke-keys"
Jul 28 14:57:30.933631 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.933636    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aceb6ade-22e2-48e5-8d57-f998a611c0c2" containerName="maas-api"
Jul 28 14:57:30.936942 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.936924    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"
Jul 28 14:57:30.939415 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.939395    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:57:30.946505 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:30.946479    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"]
Jul 28 14:57:31.006955 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:31.006916    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f03df55-cd52-467c-866f-d4a4a32ed6c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9\" (UID: \"4f03df55-cd52-467c-866f-d4a4a32ed6c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"
Jul 28 14:57:31.107541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:31.107504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f03df55-cd52-467c-866f-d4a4a32ed6c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9\" (UID: \"4f03df55-cd52-467c-866f-d4a4a32ed6c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"
Jul 28 14:57:31.108137 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:31.108119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f03df55-cd52-467c-866f-d4a4a32ed6c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9\" (UID: \"4f03df55-cd52-467c-866f-d4a4a32ed6c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"
Jul 28 14:57:31.248353 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:31.248272    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"
Jul 28 14:57:31.389469 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:31.389438    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"]
Jul 28 14:57:31.390896 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:57:31.390872    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f03df55_cd52_467c_866f_d4a4a32ed6c4.slice/crio-4cf359851ec39b45fda76cfe8800b0644f21fe8760f940a32afe833a120048b4 WatchSource:0}: Error finding container 4cf359851ec39b45fda76cfe8800b0644f21fe8760f940a32afe833a120048b4: Status 404 returned error can't find the container with id 4cf359851ec39b45fda76cfe8800b0644f21fe8760f940a32afe833a120048b4
Jul 28 14:57:32.373147 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:32.373064    2579 generic.go:356] "Generic (PLEG): container finished" podID="4f03df55-cd52-467c-866f-d4a4a32ed6c4" containerID="3686e3b3874eb56f86788af318c69d94daed8969b01362d2e390dfa69ff9df4d" exitCode=0
Jul 28 14:57:32.373147 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:32.373136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9" event={"ID":"4f03df55-cd52-467c-866f-d4a4a32ed6c4","Type":"ContainerDied","Data":"3686e3b3874eb56f86788af318c69d94daed8969b01362d2e390dfa69ff9df4d"}
Jul 28 14:57:32.373553 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:32.373176    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9" event={"ID":"4f03df55-cd52-467c-866f-d4a4a32ed6c4","Type":"ContainerStarted","Data":"4cf359851ec39b45fda76cfe8800b0644f21fe8760f940a32afe833a120048b4"}
Jul 28 14:57:33.510787 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:33.510764    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"
Jul 28 14:57:33.632724 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:33.632642    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f03df55-cd52-467c-866f-d4a4a32ed6c4-maas-api-service-ca\") pod \"4f03df55-cd52-467c-866f-d4a4a32ed6c4\" (UID: \"4f03df55-cd52-467c-866f-d4a4a32ed6c4\") "
Jul 28 14:57:33.632986 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:33.632964    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f03df55-cd52-467c-866f-d4a4a32ed6c4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f03df55-cd52-467c-866f-d4a4a32ed6c4" (UID: "4f03df55-cd52-467c-866f-d4a4a32ed6c4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:57:33.733409 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:33.733378    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f03df55-cd52-467c-866f-d4a4a32ed6c4-maas-api-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:57:34.386521 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:34.386488    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9" event={"ID":"4f03df55-cd52-467c-866f-d4a4a32ed6c4","Type":"ContainerDied","Data":"4cf359851ec39b45fda76cfe8800b0644f21fe8760f940a32afe833a120048b4"}
Jul 28 14:57:34.386682 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:34.386533    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cf359851ec39b45fda76cfe8800b0644f21fe8760f940a32afe833a120048b4"
Jul 28 14:57:34.386682 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:34.386502    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"
Jul 28 14:57:38.325999 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.325961    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"]
Jul 28 14:57:38.326499 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.326251    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8" podUID="b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c" containerName="maas-api" containerID="cri-o://8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a" gracePeriod=30
Jul 28 14:57:38.562731 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.562698    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"]
Jul 28 14:57:38.564987 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.564966    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fd71df-shlp9"]
Jul 28 14:57:38.573924 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.573900    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:57:38.674839 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.674747    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-maas-api-tls\") pod \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") "
Jul 28 14:57:38.674839 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.674794    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-openshift-service-ca\") pod \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") "
Jul 28 14:57:38.674839 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.674833    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nzt2\" (UniqueName: \"kubernetes.io/projected/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-kube-api-access-6nzt2\") pod \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\" (UID: \"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c\") "
Jul 28 14:57:38.675236 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.675215    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c" (UID: "b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:57:38.677169 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.677139    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-kube-api-access-6nzt2" (OuterVolumeSpecName: "kube-api-access-6nzt2") pod "b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c" (UID: "b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c"). InnerVolumeSpecName "kube-api-access-6nzt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:57:38.677270 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.677177    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c" (UID: "b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:57:38.776235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.776197    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nzt2\" (UniqueName: \"kubernetes.io/projected/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-kube-api-access-6nzt2\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:57:38.776235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.776230    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-maas-api-tls\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:57:38.776235 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:38.776240    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c-openshift-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:57:39.416478 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:39.416440    2579 generic.go:356] "Generic (PLEG): container finished" podID="b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c" containerID="8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a" exitCode=0
Jul 28 14:57:39.416478 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:39.416483    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8" event={"ID":"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c","Type":"ContainerDied","Data":"8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a"}
Jul 28 14:57:39.417055 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:39.416487    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"
Jul 28 14:57:39.417055 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:39.416538    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8" event={"ID":"b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c","Type":"ContainerDied","Data":"eb15c3811a0ebba05ff42f2a3f997400bf22b9029602dc47e0fe6b4f7326406e"}
Jul 28 14:57:39.417055 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:39.416567    2579 scope.go:117] "RemoveContainer" containerID="8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a"
Jul 28 14:57:39.426486 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:39.426468    2579 scope.go:117] "RemoveContainer" containerID="8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a"
Jul 28 14:57:39.426736 ip-10-0-143-3 kubenswrapper[2579]: E0728 14:57:39.426717    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a\": container with ID starting with 8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a not found: ID does not exist" containerID="8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a"
Jul 28 14:57:39.426782 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:39.426746    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a"} err="failed to get container status \"8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a\": rpc error: code = NotFound desc = could not find container \"8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a\": container with ID starting with 8c590d2c4656d3aaa5e3ccc2e3106c4cd930e57980ea2da8fe01c0c525b9f92a not found: ID does not exist"
Jul 28 14:57:39.439039 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:39.438990    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"]
Jul 28 14:57:39.441633 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:39.441608    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fd71df-57676fc77c-f82h8"]
Jul 28 14:57:40.341659 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:40.341631    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f03df55-cd52-467c-866f-d4a4a32ed6c4" path="/var/lib/kubelet/pods/4f03df55-cd52-467c-866f-d4a4a32ed6c4/volumes"
Jul 28 14:57:40.342008 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:40.341995    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c" path="/var/lib/kubelet/pods/b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c/volumes"
Jul 28 14:57:41.838510 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:41.838479    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:57:46.712541 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.712504    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"]
Jul 28 14:57:46.712937 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.712916    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c" containerName="maas-api"
Jul 28 14:57:46.712937 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.712928    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c" containerName="maas-api"
Jul 28 14:57:46.713048 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.712941    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f03df55-cd52-467c-866f-d4a4a32ed6c4" containerName="revoke-keys"
Jul 28 14:57:46.713048 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.712947    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f03df55-cd52-467c-866f-d4a4a32ed6c4" containerName="revoke-keys"
Jul 28 14:57:46.713048 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.713042    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f03df55-cd52-467c-866f-d4a4a32ed6c4" containerName="revoke-keys"
Jul 28 14:57:46.713156 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.713064    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7bcd0e8-ceba-4dfa-913e-c2bbd7331b1c" containerName="maas-api"
Jul 28 14:57:46.717477 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.717460    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"
Jul 28 14:57:46.719806 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.719776    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 14:57:46.719806 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.719801    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:57:46.724766 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.724742    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"]
Jul 28 14:57:46.848033 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.847992    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/185fa21a-9464-44d6-b6af-572e6b914405-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-910fb6-497w8\" (UID: \"185fa21a-9464-44d6-b6af-572e6b914405\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"
Jul 28 14:57:46.949164 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.949124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/185fa21a-9464-44d6-b6af-572e6b914405-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-910fb6-497w8\" (UID: \"185fa21a-9464-44d6-b6af-572e6b914405\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"
Jul 28 14:57:46.949764 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:46.949745    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/185fa21a-9464-44d6-b6af-572e6b914405-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-910fb6-497w8\" (UID: \"185fa21a-9464-44d6-b6af-572e6b914405\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"
Jul 28 14:57:47.030279 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:47.030256    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"
Jul 28 14:57:47.169816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:47.169785    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"]
Jul 28 14:57:47.170243 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:57:47.170216    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185fa21a_9464_44d6_b6af_572e6b914405.slice/crio-3f537657e7c3ffaf233e421f1523108702b9eddbb9a430a951fb13fa87aaf7bb WatchSource:0}: Error finding container 3f537657e7c3ffaf233e421f1523108702b9eddbb9a430a951fb13fa87aaf7bb: Status 404 returned error can't find the container with id 3f537657e7c3ffaf233e421f1523108702b9eddbb9a430a951fb13fa87aaf7bb
Jul 28 14:57:47.463196 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:47.463154    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8" event={"ID":"185fa21a-9464-44d6-b6af-572e6b914405","Type":"ContainerStarted","Data":"798637197c6570216f5d370ef002bbdca2eb77f615585b11e0639f5ace43f7c5"}
Jul 28 14:57:47.463196 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:47.463203    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8" event={"ID":"185fa21a-9464-44d6-b6af-572e6b914405","Type":"ContainerStarted","Data":"3f537657e7c3ffaf233e421f1523108702b9eddbb9a430a951fb13fa87aaf7bb"}
Jul 28 14:57:47.484297 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:47.484251    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8" podStartSLOduration=1.484235816 podStartE2EDuration="1.484235816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:57:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:57:47.482804079 +0000 UTC m=+3483.610194770" watchObservedRunningTime="2026-07-28 14:57:47.484235816 +0000 UTC m=+3483.611626507"
Jul 28 14:57:48.470797 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:48.470763    2579 generic.go:356] "Generic (PLEG): container finished" podID="185fa21a-9464-44d6-b6af-572e6b914405" containerID="798637197c6570216f5d370ef002bbdca2eb77f615585b11e0639f5ace43f7c5" exitCode=0
Jul 28 14:57:48.471241 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:48.470855    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8" event={"ID":"185fa21a-9464-44d6-b6af-572e6b914405","Type":"ContainerDied","Data":"798637197c6570216f5d370ef002bbdca2eb77f615585b11e0639f5ace43f7c5"}
Jul 28 14:57:49.609778 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:49.609754    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"
Jul 28 14:57:49.672573 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:49.672543    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/185fa21a-9464-44d6-b6af-572e6b914405-maas-api-service-ca\") pod \"185fa21a-9464-44d6-b6af-572e6b914405\" (UID: \"185fa21a-9464-44d6-b6af-572e6b914405\") "
Jul 28 14:57:49.672906 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:49.672884    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/185fa21a-9464-44d6-b6af-572e6b914405-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "185fa21a-9464-44d6-b6af-572e6b914405" (UID: "185fa21a-9464-44d6-b6af-572e6b914405"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:57:49.773916 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:49.773885    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/185fa21a-9464-44d6-b6af-572e6b914405-maas-api-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:57:50.483500 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:50.483470    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"
Jul 28 14:57:50.483669 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:50.483511    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8" event={"ID":"185fa21a-9464-44d6-b6af-572e6b914405","Type":"ContainerDied","Data":"3f537657e7c3ffaf233e421f1523108702b9eddbb9a430a951fb13fa87aaf7bb"}
Jul 28 14:57:50.483669 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:50.483553    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f537657e7c3ffaf233e421f1523108702b9eddbb9a430a951fb13fa87aaf7bb"
Jul 28 14:57:52.591998 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:52.591943    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t"]
Jul 28 14:57:52.594386 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:52.594360    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-910fb6-29754165-2ld2t"]
Jul 28 14:57:52.839387 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:52.839350    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"]
Jul 28 14:57:52.845040 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:52.844948    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-910fb6-497w8"]
Jul 28 14:57:54.342579 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:54.342545    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b103b6c-b3e2-4140-870c-889f590a2c52" path="/var/lib/kubelet/pods/0b103b6c-b3e2-4140-870c-889f590a2c52/volumes"
Jul 28 14:57:54.342974 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:54.342841    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="185fa21a-9464-44d6-b6af-572e6b914405" path="/var/lib/kubelet/pods/185fa21a-9464-44d6-b6af-572e6b914405/volumes"
Jul 28 14:57:55.643130 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:57:55.643098    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:58:01.579872 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.579839    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"]
Jul 28 14:58:01.580275 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.580239    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="185fa21a-9464-44d6-b6af-572e6b914405" containerName="revoke-keys"
Jul 28 14:58:01.580275 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.580258    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="185fa21a-9464-44d6-b6af-572e6b914405" containerName="revoke-keys"
Jul 28 14:58:01.580360 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.580334    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="185fa21a-9464-44d6-b6af-572e6b914405" containerName="revoke-keys"
Jul 28 14:58:01.583435 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.583418    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"
Jul 28 14:58:01.585744 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.585721    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 14:58:01.585848 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.585794    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s8tsp\""
Jul 28 14:58:01.590756 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.590721    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"]
Jul 28 14:58:01.685271 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.685238    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b65415-5a64-4d15-ba11-caa0a093236e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-baa216-lz96n\" (UID: \"e3b65415-5a64-4d15-ba11-caa0a093236e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"
Jul 28 14:58:01.785779 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.785746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b65415-5a64-4d15-ba11-caa0a093236e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-baa216-lz96n\" (UID: \"e3b65415-5a64-4d15-ba11-caa0a093236e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"
Jul 28 14:58:01.786554 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.786530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b65415-5a64-4d15-ba11-caa0a093236e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-baa216-lz96n\" (UID: \"e3b65415-5a64-4d15-ba11-caa0a093236e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"
Jul 28 14:58:01.895409 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:01.895330    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"
Jul 28 14:58:02.033111 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:02.033082    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"]
Jul 28 14:58:02.035184 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:58:02.035152    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b65415_5a64_4d15_ba11_caa0a093236e.slice/crio-961868fe21faf4a287f79c04bbc5a80ee0f738837454ede01b0983560ff54b18 WatchSource:0}: Error finding container 961868fe21faf4a287f79c04bbc5a80ee0f738837454ede01b0983560ff54b18: Status 404 returned error can't find the container with id 961868fe21faf4a287f79c04bbc5a80ee0f738837454ede01b0983560ff54b18
Jul 28 14:58:02.564869 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:02.564830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n" event={"ID":"e3b65415-5a64-4d15-ba11-caa0a093236e","Type":"ContainerStarted","Data":"c6b8e0d1be034e2bc91a651e964ed5d0ef29b151e164edc0695b7e844e530327"}
Jul 28 14:58:02.565080 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:02.564891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n" event={"ID":"e3b65415-5a64-4d15-ba11-caa0a093236e","Type":"ContainerStarted","Data":"961868fe21faf4a287f79c04bbc5a80ee0f738837454ede01b0983560ff54b18"}
Jul 28 14:58:02.583085 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:02.583030    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n" podStartSLOduration=1.582998567 podStartE2EDuration="1.582998567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:58:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:58:02.58133464 +0000 UTC m=+3498.708725335" watchObservedRunningTime="2026-07-28 14:58:02.582998567 +0000 UTC m=+3498.710389257"
Jul 28 14:58:03.572559 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:03.572523    2579 generic.go:356] "Generic (PLEG): container finished" podID="e3b65415-5a64-4d15-ba11-caa0a093236e" containerID="c6b8e0d1be034e2bc91a651e964ed5d0ef29b151e164edc0695b7e844e530327" exitCode=0
Jul 28 14:58:03.572739 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:03.572606    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n" event={"ID":"e3b65415-5a64-4d15-ba11-caa0a093236e","Type":"ContainerDied","Data":"c6b8e0d1be034e2bc91a651e964ed5d0ef29b151e164edc0695b7e844e530327"}
Jul 28 14:58:04.712245 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:04.712213    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"
Jul 28 14:58:04.809758 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:04.809680    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b65415-5a64-4d15-ba11-caa0a093236e-maas-api-service-ca\") pod \"e3b65415-5a64-4d15-ba11-caa0a093236e\" (UID: \"e3b65415-5a64-4d15-ba11-caa0a093236e\") "
Jul 28 14:58:04.809995 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:04.809971    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3b65415-5a64-4d15-ba11-caa0a093236e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e3b65415-5a64-4d15-ba11-caa0a093236e" (UID: "e3b65415-5a64-4d15-ba11-caa0a093236e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:58:04.910764 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:04.910686    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b65415-5a64-4d15-ba11-caa0a093236e-maas-api-service-ca\") on node \"ip-10-0-143-3.ec2.internal\" DevicePath \"\""
Jul 28 14:58:05.585720 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:05.585690    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"
Jul 28 14:58:05.585879 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:05.585722    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n" event={"ID":"e3b65415-5a64-4d15-ba11-caa0a093236e","Type":"ContainerDied","Data":"961868fe21faf4a287f79c04bbc5a80ee0f738837454ede01b0983560ff54b18"}
Jul 28 14:58:05.585879 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:05.585794    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="961868fe21faf4a287f79c04bbc5a80ee0f738837454ede01b0983560ff54b18"
Jul 28 14:58:06.947143 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:06.947096    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"]
Jul 28 14:58:06.949369 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:06.949350    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-baa216-lz96n"]
Jul 28 14:58:08.342490 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:08.342457    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3b65415-5a64-4d15-ba11-caa0a093236e" path="/var/lib/kubelet/pods/e3b65415-5a64-4d15-ba11-caa0a093236e/volumes"
Jul 28 14:58:09.635347 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:09.635313    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zrc7c"]
Jul 28 14:58:14.998709 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:14.998674    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8587bdf57d-gck8j_4c61ce58-9932-4032-b998-6b8eacfd5834/authorino/0.log"
Jul 28 14:58:20.945747 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:20.945718    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lj2hf_0b2609ab-ae57-4514-a372-1c05a74b180b/server/0.log"
Jul 28 14:58:21.067408 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:21.067381    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-d8ctb_870a4744-f59d-40ea-8b33-ce48af9afa40/manager/0.log"
Jul 28 14:58:21.308134 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:21.308105    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b59958787-mpjlj_062f3464-4431-4919-ac0c-d8b2a8b5b562/copy-manifests/0.log"
Jul 28 14:58:21.317042 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:21.316999    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b59958787-mpjlj_062f3464-4431-4919-ac0c-d8b2a8b5b562/manager/0.log"
Jul 28 14:58:22.148705 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:22.148677    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754165-zkkbf_0807e135-8a65-4961-a701-d033af08784a/cleanup/0.log"
Jul 28 14:58:23.041056 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.041004    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c_1ca6eeb4-4269-4ec0-a231-da61c5816719/util/0.log"
Jul 28 14:58:23.049120 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.049096    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c_1ca6eeb4-4269-4ec0-a231-da61c5816719/pull/0.log"
Jul 28 14:58:23.056783 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.056765    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c_1ca6eeb4-4269-4ec0-a231-da61c5816719/extract/0.log"
Jul 28 14:58:23.176668 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.176640    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c_792d6e64-1728-48ea-840f-228c68117c27/util/0.log"
Jul 28 14:58:23.188720 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.188695    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c_792d6e64-1728-48ea-840f-228c68117c27/pull/0.log"
Jul 28 14:58:23.207530 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.207506    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c_792d6e64-1728-48ea-840f-228c68117c27/extract/0.log"
Jul 28 14:58:23.333902 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.333826    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5_c1cf73ab-75f2-4779-b3da-2a0f653ddb40/pull/0.log"
Jul 28 14:58:23.342106 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.342085    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5_c1cf73ab-75f2-4779-b3da-2a0f653ddb40/extract/0.log"
Jul 28 14:58:23.350275 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.350254    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5_c1cf73ab-75f2-4779-b3da-2a0f653ddb40/util/0.log"
Jul 28 14:58:23.462233 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.462209    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt_54c9af46-5b09-4b16-a585-3de8baa9d6b9/pull/0.log"
Jul 28 14:58:23.469819 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.469795    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt_54c9af46-5b09-4b16-a585-3de8baa9d6b9/extract/0.log"
Jul 28 14:58:23.477060 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.477038    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt_54c9af46-5b09-4b16-a585-3de8baa9d6b9/util/0.log"
Jul 28 14:58:23.596369 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.596293    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8587bdf57d-gck8j_4c61ce58-9932-4032-b998-6b8eacfd5834/authorino/0.log"
Jul 28 14:58:23.945973 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:23.945893    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5xx2m_89d3b87e-94de-4b39-a9a3-0b309bc02fd5/kuadrant-console-plugin/0.log"
Jul 28 14:58:24.330876 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:24.330850    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zrc7c_015b1470-ed09-4093-8e1a-373d705b7061/limitador/0.log"
Jul 28 14:58:25.065917 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:25.065889    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-97487846c-7xmvp_ef918895-4378-4d1d-8d5d-4f47f4595333/kube-auth-proxy/0.log"
Jul 28 14:58:25.664105 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:25.664073    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c97498d97-j9zq7_0a4f01d9-d677-4b3c-8d48-7076c851f2fd/router/0.log"
Jul 28 14:58:32.440154 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.440113    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qh668/must-gather-2c5xn"]
Jul 28 14:58:32.440835 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.440814    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3b65415-5a64-4d15-ba11-caa0a093236e" containerName="revoke-keys"
Jul 28 14:58:32.440929 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.440838    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3b65415-5a64-4d15-ba11-caa0a093236e" containerName="revoke-keys"
Jul 28 14:58:32.440990 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.440952    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3b65415-5a64-4d15-ba11-caa0a093236e" containerName="revoke-keys"
Jul 28 14:58:32.444281 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.444260    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh668/must-gather-2c5xn"
Jul 28 14:58:32.446816 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.446796    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qh668\"/\"default-dockercfg-gjnsx\""
Jul 28 14:58:32.446947 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.446845    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qh668\"/\"openshift-service-ca.crt\""
Jul 28 14:58:32.447102 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.446950    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qh668\"/\"kube-root-ca.crt\""
Jul 28 14:58:32.454869 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.454849    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh668/must-gather-2c5xn"]
Jul 28 14:58:32.552704 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.552668    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/08bc4f88-6920-43a7-83ae-74e7e5a2d879-must-gather-output\") pod \"must-gather-2c5xn\" (UID: \"08bc4f88-6920-43a7-83ae-74e7e5a2d879\") " pod="openshift-must-gather-qh668/must-gather-2c5xn"
Jul 28 14:58:32.552868 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.552708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkchz\" (UniqueName: \"kubernetes.io/projected/08bc4f88-6920-43a7-83ae-74e7e5a2d879-kube-api-access-pkchz\") pod \"must-gather-2c5xn\" (UID: \"08bc4f88-6920-43a7-83ae-74e7e5a2d879\") " pod="openshift-must-gather-qh668/must-gather-2c5xn"
Jul 28 14:58:32.653453 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.653419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/08bc4f88-6920-43a7-83ae-74e7e5a2d879-must-gather-output\") pod \"must-gather-2c5xn\" (UID: \"08bc4f88-6920-43a7-83ae-74e7e5a2d879\") " pod="openshift-must-gather-qh668/must-gather-2c5xn"
Jul 28 14:58:32.653453 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.653460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkchz\" (UniqueName: \"kubernetes.io/projected/08bc4f88-6920-43a7-83ae-74e7e5a2d879-kube-api-access-pkchz\") pod \"must-gather-2c5xn\" (UID: \"08bc4f88-6920-43a7-83ae-74e7e5a2d879\") " pod="openshift-must-gather-qh668/must-gather-2c5xn"
Jul 28 14:58:32.653772 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.653751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/08bc4f88-6920-43a7-83ae-74e7e5a2d879-must-gather-output\") pod \"must-gather-2c5xn\" (UID: \"08bc4f88-6920-43a7-83ae-74e7e5a2d879\") " pod="openshift-must-gather-qh668/must-gather-2c5xn"
Jul 28 14:58:32.662221 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.662198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkchz\" (UniqueName: \"kubernetes.io/projected/08bc4f88-6920-43a7-83ae-74e7e5a2d879-kube-api-access-pkchz\") pod \"must-gather-2c5xn\" (UID: \"08bc4f88-6920-43a7-83ae-74e7e5a2d879\") " pod="openshift-must-gather-qh668/must-gather-2c5xn"
Jul 28 14:58:32.754277 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.754182    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh668/must-gather-2c5xn"
Jul 28 14:58:32.891608 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.891578    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh668/must-gather-2c5xn"]
Jul 28 14:58:32.893317 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:58:32.893283    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08bc4f88_6920_43a7_83ae_74e7e5a2d879.slice/crio-02fc24a36b0745c7f7005cba0c79796c9966d6d62bac7e9051448d6078ddc35e WatchSource:0}: Error finding container 02fc24a36b0745c7f7005cba0c79796c9966d6d62bac7e9051448d6078ddc35e: Status 404 returned error can't find the container with id 02fc24a36b0745c7f7005cba0c79796c9966d6d62bac7e9051448d6078ddc35e
Jul 28 14:58:32.895320 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:32.895303    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:58:33.748683 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:33.748658    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh668/must-gather-2c5xn" event={"ID":"08bc4f88-6920-43a7-83ae-74e7e5a2d879","Type":"ContainerStarted","Data":"02fc24a36b0745c7f7005cba0c79796c9966d6d62bac7e9051448d6078ddc35e"}
Jul 28 14:58:34.757978 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:34.757942    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh668/must-gather-2c5xn" event={"ID":"08bc4f88-6920-43a7-83ae-74e7e5a2d879","Type":"ContainerStarted","Data":"04086a1c7f89856b0f21735d9b38ff66cdfaf0f45aa0dfc26063b0fbdc50ff15"}
Jul 28 14:58:34.757978 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:34.757986    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh668/must-gather-2c5xn" event={"ID":"08bc4f88-6920-43a7-83ae-74e7e5a2d879","Type":"ContainerStarted","Data":"c623452456897e2bd30d3ad9d71a68226c4fd64a7f8d2fb185a6455ffcd3afaf"}
Jul 28 14:58:34.776300 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:34.776240    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qh668/must-gather-2c5xn" podStartSLOduration=1.9257203 podStartE2EDuration="2.776217803s" totalImagesPullingTime="850.497503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:58:34.774344155 +0000 UTC m=+3530.901734847" watchObservedRunningTime="2026-07-28 14:58:34.776217803 +0000 UTC m=+3530.903608497"
Jul 28 14:58:35.525069 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:35.525037    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c75dk_e2f0a8fb-0a74-4e58-8195-47b0ac92551b/global-pull-secret-syncer/0.log"
Jul 28 14:58:35.618568 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:35.618531    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jgqhc_2f4b440f-f03f-4c5e-ad0b-0935b299d316/konnectivity-agent/0.log"
Jul 28 14:58:35.737455 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:35.737422    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-3.ec2.internal_6180e30e1977ed4d8d23fa6e6c33c1ed/haproxy/0.log"
Jul 28 14:58:39.852468 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:39.852424    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c_1ca6eeb4-4269-4ec0-a231-da61c5816719/extract/0.log"
Jul 28 14:58:39.876193 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:39.876164    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c_1ca6eeb4-4269-4ec0-a231-da61c5816719/util/0.log"
Jul 28 14:58:39.912611 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:39.912568    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h4q8c_1ca6eeb4-4269-4ec0-a231-da61c5816719/pull/0.log"
Jul 28 14:58:39.944484 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:39.944417    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c_792d6e64-1728-48ea-840f-228c68117c27/extract/0.log"
Jul 28 14:58:39.971348 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:39.971326    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c_792d6e64-1728-48ea-840f-228c68117c27/util/0.log"
Jul 28 14:58:39.997135 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:39.997072    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05fj6c_792d6e64-1728-48ea-840f-228c68117c27/pull/0.log"
Jul 28 14:58:40.051197 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:40.051170    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5_c1cf73ab-75f2-4779-b3da-2a0f653ddb40/extract/0.log"
Jul 28 14:58:40.078101 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:40.078067    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5_c1cf73ab-75f2-4779-b3da-2a0f653ddb40/util/0.log"
Jul 28 14:58:40.102525 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:40.102443    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q7bn5_c1cf73ab-75f2-4779-b3da-2a0f653ddb40/pull/0.log"
Jul 28 14:58:40.138963 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:40.138937    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt_54c9af46-5b09-4b16-a585-3de8baa9d6b9/extract/0.log"
Jul 28 14:58:40.164442 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:40.164354    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt_54c9af46-5b09-4b16-a585-3de8baa9d6b9/util/0.log"
Jul 28 14:58:40.198357 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:40.198325    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bfxlt_54c9af46-5b09-4b16-a585-3de8baa9d6b9/pull/0.log"
Jul 28 14:58:40.721043 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:40.718257    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8587bdf57d-gck8j_4c61ce58-9932-4032-b998-6b8eacfd5834/authorino/0.log"
Jul 28 14:58:40.817476 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:40.817448    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5xx2m_89d3b87e-94de-4b39-a9a3-0b309bc02fd5/kuadrant-console-plugin/0.log"
Jul 28 14:58:41.079274 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:41.079240    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zrc7c_015b1470-ed09-4093-8e1a-373d705b7061/limitador/0.log"
Jul 28 14:58:42.938206 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:42.938096    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7524694e-0ec3-469f-af9c-bdfb7158eb3a/alertmanager/0.log"
Jul 28 14:58:42.965919 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:42.965885    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7524694e-0ec3-469f-af9c-bdfb7158eb3a/config-reloader/0.log"
Jul 28 14:58:42.993290 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:42.993255    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7524694e-0ec3-469f-af9c-bdfb7158eb3a/kube-rbac-proxy-web/0.log"
Jul 28 14:58:43.021761 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.021719    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7524694e-0ec3-469f-af9c-bdfb7158eb3a/kube-rbac-proxy/0.log"
Jul 28 14:58:43.048752 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.048713    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7524694e-0ec3-469f-af9c-bdfb7158eb3a/kube-rbac-proxy-metric/0.log"
Jul 28 14:58:43.075374 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.075336    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7524694e-0ec3-469f-af9c-bdfb7158eb3a/prom-label-proxy/0.log"
Jul 28 14:58:43.103392 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.103347    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7524694e-0ec3-469f-af9c-bdfb7158eb3a/init-config-reloader/0.log"
Jul 28 14:58:43.151777 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.151715    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-2rhml_97a45da1-bd05-4efe-b270-1f7a8bff838e/cluster-monitoring-operator/0.log"
Jul 28 14:58:43.182716 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.182687    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7w4jd_723bc418-d780-4e89-b5ad-89dbe94ffda5/kube-state-metrics/0.log"
Jul 28 14:58:43.206918 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.206843    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7w4jd_723bc418-d780-4e89-b5ad-89dbe94ffda5/kube-rbac-proxy-main/0.log"
Jul 28 14:58:43.239545 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.239519    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7w4jd_723bc418-d780-4e89-b5ad-89dbe94ffda5/kube-rbac-proxy-self/0.log"
Jul 28 14:58:43.305434 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.305399    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-5mftw_42c4905e-b1cb-4a00-85c0-d68bf95ec142/monitoring-plugin/0.log"
Jul 28 14:58:43.338380 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.338303    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-52cwh_63d71a17-a444-4fba-af29-fe4217c8c252/node-exporter/0.log"
Jul 28 14:58:43.362744 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.362712    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-52cwh_63d71a17-a444-4fba-af29-fe4217c8c252/kube-rbac-proxy/0.log"
Jul 28 14:58:43.390355 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.390321    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-52cwh_63d71a17-a444-4fba-af29-fe4217c8c252/init-textfile/0.log"
Jul 28 14:58:43.935810 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.935782    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64775789b7-6rkb8_ba905d8d-911d-4b94-ad05-50927a13c5af/telemeter-client/0.log"
Jul 28 14:58:43.960708 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.960614    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64775789b7-6rkb8_ba905d8d-911d-4b94-ad05-50927a13c5af/reload/0.log"
Jul 28 14:58:43.986117 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:43.986091    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64775789b7-6rkb8_ba905d8d-911d-4b94-ad05-50927a13c5af/kube-rbac-proxy/0.log"
Jul 28 14:58:44.022750 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:44.022722    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f984bcdc7-qjj8h_91b2b8f5-583d-4396-bbdb-0c96b3150ea3/thanos-query/0.log"
Jul 28 14:58:44.047904 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:44.047879    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f984bcdc7-qjj8h_91b2b8f5-583d-4396-bbdb-0c96b3150ea3/kube-rbac-proxy-web/0.log"
Jul 28 14:58:44.080052 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:44.079985    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f984bcdc7-qjj8h_91b2b8f5-583d-4396-bbdb-0c96b3150ea3/kube-rbac-proxy/0.log"
Jul 28 14:58:44.104468 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:44.104437    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f984bcdc7-qjj8h_91b2b8f5-583d-4396-bbdb-0c96b3150ea3/prom-label-proxy/0.log"
Jul 28 14:58:44.129038 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:44.128991    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f984bcdc7-qjj8h_91b2b8f5-583d-4396-bbdb-0c96b3150ea3/kube-rbac-proxy-rules/0.log"
Jul 28 14:58:44.155297 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:44.155269    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f984bcdc7-qjj8h_91b2b8f5-583d-4396-bbdb-0c96b3150ea3/kube-rbac-proxy-metrics/0.log"
Jul 28 14:58:45.481639 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.481601    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"]
Jul 28 14:58:45.491559 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.491528    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.493703 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.493668    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"]
Jul 28 14:58:45.508587 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.508557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jtp9\" (UniqueName: \"kubernetes.io/projected/b17d7829-eee9-4d27-ba5b-2ce23d36973b-kube-api-access-8jtp9\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.508865 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.508828    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-podres\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.509075 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.509055    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-lib-modules\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.509223 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.509208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-sys\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.509369 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.509354    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-proc\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.610376 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.610338    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jtp9\" (UniqueName: \"kubernetes.io/projected/b17d7829-eee9-4d27-ba5b-2ce23d36973b-kube-api-access-8jtp9\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.610562 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.610422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-podres\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.610562 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.610468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-lib-modules\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.610562 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.610497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-sys\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.610562 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.610542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-proc\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.610789 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.610635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-proc\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.611088 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.611062    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-podres\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.611199 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.611167    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-lib-modules\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.611259 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.611208    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b17d7829-eee9-4d27-ba5b-2ce23d36973b-sys\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.620339 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.620314    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jtp9\" (UniqueName: \"kubernetes.io/projected/b17d7829-eee9-4d27-ba5b-2ce23d36973b-kube-api-access-8jtp9\") pod \"perf-node-gather-daemonset-nfsdx\" (UID: \"b17d7829-eee9-4d27-ba5b-2ce23d36973b\") " pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.805234 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.805200    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:45.993355 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:45.993193    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"]
Jul 28 14:58:45.997040 ip-10-0-143-3 kubenswrapper[2579]: W0728 14:58:45.995782    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb17d7829_eee9_4d27_ba5b_2ce23d36973b.slice/crio-d72cd776871fc3ee7b3d63f7115e12da090cc374797208299bf708400894010b WatchSource:0}: Error finding container d72cd776871fc3ee7b3d63f7115e12da090cc374797208299bf708400894010b: Status 404 returned error can't find the container with id d72cd776871fc3ee7b3d63f7115e12da090cc374797208299bf708400894010b
Jul 28 14:58:46.043929 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:46.043298    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/2.log"
Jul 28 14:58:46.050967 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:46.050940    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-nhmps_70a294fd-3ee6-40f5-b772-5b4d8dca14fd/console-operator/3.log"
Jul 28 14:58:46.865489 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:46.865442    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx" event={"ID":"b17d7829-eee9-4d27-ba5b-2ce23d36973b","Type":"ContainerStarted","Data":"43922516d0ecf80ca49af7ad51844d7027cf6f5b05d627842713b0e0bc9fb3a7"}
Jul 28 14:58:46.865983 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:46.865520    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx" event={"ID":"b17d7829-eee9-4d27-ba5b-2ce23d36973b","Type":"ContainerStarted","Data":"d72cd776871fc3ee7b3d63f7115e12da090cc374797208299bf708400894010b"}
Jul 28 14:58:46.865983 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:46.865785    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:46.886362 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:46.886304    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx" podStartSLOduration=1.8862808370000002 podStartE2EDuration="1.886280837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:58:46.882707408 +0000 UTC m=+3543.010098100" watchObservedRunningTime="2026-07-28 14:58:46.886280837 +0000 UTC m=+3543.013671530"
Jul 28 14:58:47.214143 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:47.214062    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-68ds6_0bfbbd28-0adc-4e9f-99d3-24828951bb8e/volume-data-source-validator/0.log"
Jul 28 14:58:48.419490 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:48.419450    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-khjf6_1e4ec977-d8d3-4bd6-9bb0-eb5df490604c/dns/0.log"
Jul 28 14:58:48.448229 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:48.448202    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-khjf6_1e4ec977-d8d3-4bd6-9bb0-eb5df490604c/kube-rbac-proxy/0.log"
Jul 28 14:58:48.603380 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:48.603344    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zfd6l_68fa69ca-ef93-41a4-9b62-14e2804b3e26/dns-node-resolver/0.log"
Jul 28 14:58:49.234895 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:49.234867    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9df2t_d510aeee-4e32-4319-a8c6-a47ba4ae9205/node-ca/0.log"
Jul 28 14:58:50.476116 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:50.476082    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-97487846c-7xmvp_ef918895-4378-4d1d-8d5d-4f47f4595333/kube-auth-proxy/0.log"
Jul 28 14:58:50.671693 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:50.671664    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c97498d97-j9zq7_0a4f01d9-d677-4b3c-8d48-7076c851f2fd/router/0.log"
Jul 28 14:58:51.549579 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:51.549550    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qqkdd_943d836b-9abc-4857-9eb8-ea5ae2a76793/serve-healthcheck-canary/0.log"
Jul 28 14:58:52.073671 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:52.073643    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-crr4w_c32900e0-0735-40d4-946c-05ec3618add9/insights-operator/0.log"
Jul 28 14:58:52.077518 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:52.077494    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-crr4w_c32900e0-0735-40d4-946c-05ec3618add9/insights-operator/1.log"
Jul 28 14:58:52.184126 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:52.184101    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hwkx9_7ddcc68d-ba22-42c3-83b8-09e80c6e837d/kube-rbac-proxy/0.log"
Jul 28 14:58:52.207820 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:52.207798    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hwkx9_7ddcc68d-ba22-42c3-83b8-09e80c6e837d/exporter/0.log"
Jul 28 14:58:52.232621 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:52.232593    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hwkx9_7ddcc68d-ba22-42c3-83b8-09e80c6e837d/extractor/0.log"
Jul 28 14:58:52.688523 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:52.688481    2579 scope.go:117] "RemoveContainer" containerID="51e7f76043ee0670926dc5f2754caed6a70bfa5e5a8bb15d21ed1afdebe4e3db"
Jul 28 14:58:52.885440 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:52.885405    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qh668/perf-node-gather-daemonset-nfsdx"
Jul 28 14:58:55.027294 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:55.027265    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lj2hf_0b2609ab-ae57-4514-a372-1c05a74b180b/server/0.log"
Jul 28 14:58:55.143414 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:55.143336    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-d8ctb_870a4744-f59d-40ea-8b33-ce48af9afa40/manager/0.log"
Jul 28 14:58:55.296274 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:55.296236    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b59958787-mpjlj_062f3464-4431-4919-ac0c-d8b2a8b5b562/manager/0.log"
Jul 28 14:58:55.327081 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:55.327054    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b59958787-mpjlj_062f3464-4431-4919-ac0c-d8b2a8b5b562/copy-manifests/0.log"
Jul 28 14:58:56.133461 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:56.133436    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-97487846c-7xmvp_ef918895-4378-4d1d-8d5d-4f47f4595333/kube-auth-proxy/0.log"
Jul 28 14:58:56.260243 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:56.260214    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c97498d97-j9zq7_0a4f01d9-d677-4b3c-8d48-7076c851f2fd/router/0.log"
Jul 28 14:58:56.700511 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:56.700484    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84947845b-klwsc_d76efeb6-b8a6-423f-92e6-99400026d3c7/manager/0.log"
Jul 28 14:58:57.878516 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:57.878488    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-97487846c-7xmvp_ef918895-4378-4d1d-8d5d-4f47f4595333/kube-auth-proxy/0.log"
Jul 28 14:58:58.006706 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:58:58.006635    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c97498d97-j9zq7_0a4f01d9-d677-4b3c-8d48-7076c851f2fd/router/0.log"
Jul 28 14:59:02.224386 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:59:02.224350    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw_85a8fcc2-28d8-4534-908a-c0ea8fdf51ba/kube-storage-version-migrator-operator/1.log"
Jul 28 14:59:02.225561 ip-10-0-143-3 kubenswrapper[2579]: I0728 14:59:02.225542    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-dtggw_85a8fcc2-28d8-4534-908a-c0ea8fdf51ba/kube-storage-version-migrator-operator/0.log"