Jul 22 12:52:00.730108 ip-10-0-140-133 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 22 12:52:00.730120 ip-10-0-140-133 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 22 12:52:00.730129 ip-10-0-140-133 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 22 12:52:00.730437 ip-10-0-140-133 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 22 12:52:10.928955 ip-10-0-140-133 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 22 12:52:10.928975 ip-10-0-140-133 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7c56a09eb16e4413836530537cddf858 --
Jul 22 12:54:07.592241 ip-10-0-140-133 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 12:54:08.677095 ip-10-0-140-133 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:54:08.677095 ip-10-0-140-133 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 12:54:08.677095 ip-10-0-140-133 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:54:08.677095 ip-10-0-140-133 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 12:54:08.677095 ip-10-0-140-133 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:54:08.678186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.678036    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 12:54:08.682889 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682872    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:54:08.682889 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682890    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682894    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682897    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682900    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682904    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682907    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682910    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682913    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682916    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682919    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682927    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682930    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682933    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682936    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682939    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682942    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682944    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682948    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682951    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682954    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:54:08.682947 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682956    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682959    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682962    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682966    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682968    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682971    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682974    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682978    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682980    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682983    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682986    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682989    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682991    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682994    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682997    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.682999    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683017    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683021    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683025    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683029    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:54:08.683456 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683035    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683040    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683043    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683047    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683049    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683052    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683055    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683058    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683061    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683064    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683066    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683079    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683082    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683085    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683088    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683091    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683094    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683097    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683100    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683103    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:54:08.683958 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683106    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683109    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683111    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683114    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683117    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683120    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683122    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683125    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683128    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683130    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683133    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683136    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683139    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683142    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683145    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683149    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683152    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683154    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683157    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:54:08.684485 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683159    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683162    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683165    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683170    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683173    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683175    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683180    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683183    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683186    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683190    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683194    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683197    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683200    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683205    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683209    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683212    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683215    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683218    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683221    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:54:08.685027 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683975    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683982    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683985    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683988    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683991    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683994    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.683997    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684016    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684020    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684024    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684027    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684030    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684034    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684037    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684040    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684043    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684046    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684048    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684052    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684055    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:54:08.685515 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684058    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684061    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684064    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684067    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684070    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684073    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684076    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684080    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684084    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684088    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684091    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684094    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684097    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684099    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684102    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684105    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684108    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684111    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684113    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:54:08.686063 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684116    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684119    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684122    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684126    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684129    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684132    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684135    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684138    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684144    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684148    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684152    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684155    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684158    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684161    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684163    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684167    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684169    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684174    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684178    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:54:08.686564 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684181    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684184    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684188    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684191    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684194    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684197    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684200    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684202    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684205    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684208    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684211    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684213    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684216    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684220    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684223    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684225    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684229    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684233    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684236    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684239    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:54:08.687068 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684241    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684244    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684247    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684250    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684252    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684255    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684258    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684260    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684264    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684268    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684271    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684274    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684276    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684279    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684282    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684285    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684288    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684290    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684293    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684296    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:54:08.687574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684299    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684690    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684696    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684699    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684702    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684706    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684711    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684715    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684719    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684723    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684726    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684729    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684732    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684735    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684738    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684741    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684744    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684746    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684749    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684752    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:54:08.688111 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684755    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684757    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684760    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684763    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684765    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684768    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684771    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684774    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684776    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684779    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684782    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684785    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684800    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684803    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684806    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684809    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684812    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684815    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684819    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:54:08.688622 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684822    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684825    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684828    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684832    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684834    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684837    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684840    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684843    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684846    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684848    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684851    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684855    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684858    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684861    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684863    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684866    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684869    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684871    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684874    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684877    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:54:08.689136 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684879    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684882    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684885    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684888    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684891    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684893    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684896    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684899    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684901    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684904    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684907    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684912    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684916    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684919    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684922    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684925    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684929    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684932    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684936    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:54:08.689656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684940    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684943    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684946    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684950    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684953    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684956    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684959    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684962    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684965    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684968    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684971    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684974    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684977    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684979    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684982    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684985    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684988    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684991    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684993    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684996    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:54:08.690198 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.684999    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685016    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685092    2575 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685100    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685109    2575 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685116    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685122    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685127    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685135    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685140    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685144    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685147    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685151    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685155    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685158    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685161    2575 flags.go:64] FLAG: --cgroup-root=""
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685164    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685167    2575 flags.go:64] FLAG: --client-ca-file=""
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685171    2575 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685174    2575 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685195    2575 flags.go:64] FLAG: --cluster-domain=""
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685199    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685203    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685206    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 12:54:08.690716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685210    2575 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685218    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685221    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685225    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685228    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685232    2575 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685235    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685238    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685242    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685245    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685249    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685256    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685260    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685263    2575 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685266    2575 flags.go:64] FLAG: --enable-server="true"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685269    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685275    2575 flags.go:64] FLAG: --event-burst="100"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685278    2575 flags.go:64] FLAG: --event-qps="50"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685281    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685286    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685289    2575 flags.go:64] FLAG: --eviction-hard=""
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685293    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685296    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685299    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685302    2575 flags.go:64] FLAG: --eviction-soft=""
Jul 22 12:54:08.691352 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685306    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685309    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685312    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685315    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685318    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685321    2575 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685324    2575 flags.go:64] FLAG: --feature-gates=""
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685329    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685332    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685335    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685339    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685342    2575 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685345    2575 flags.go:64] FLAG: --help="false"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685349    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-140-133.ec2.internal"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685352    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685355    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685358    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685362    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685369    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685372    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685375    2575 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685378    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685381    2575 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 12:54:08.691989 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685384    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685388    2575 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685391    2575 flags.go:64] FLAG: --kube-reserved=""
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685394    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685398    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685402    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685405    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685408    2575 flags.go:64] FLAG: --lock-file=""
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685411    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685414    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685417    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685423    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685426    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685429    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685432    2575 flags.go:64] FLAG: --logging-format="text"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685435    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685439    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685442    2575 flags.go:64] FLAG: --manifest-url=""
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685445    2575 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685450    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685453    2575 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685457    2575 flags.go:64] FLAG: --max-pods="110"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685461    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685464    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685467    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 12:54:08.692957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685470    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685474    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685481    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685484    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685492    2575 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685495    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685498    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685502    2575 flags.go:64] FLAG: --pod-cidr=""
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685505    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685511    2575 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685514    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685518    2575 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685522    2575 flags.go:64] FLAG: --port="10250"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685526    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685529    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0028e57eb8533535b"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685533    2575 flags.go:64] FLAG: --qos-reserved=""
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685536    2575 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685539    2575 flags.go:64] FLAG: --register-node="true"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685542    2575 flags.go:64] FLAG: --register-with-taints=""
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685546    2575 flags.go:64] FLAG: --registry-burst="10"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685549    2575 flags.go:64] FLAG: --registry-qps="5"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685552    2575 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685555    2575 flags.go:64] FLAG: --reserved-memory=""
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685559    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685562    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 12:54:08.694296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685565    2575 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685568    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685571    2575 flags.go:64] FLAG: --runonce="false"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685574    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685579    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685582    2575 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685585    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685588    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685592    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685597    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685602    2575 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685605    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685608    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685611    2575 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685615    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685618    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685621    2575 flags.go:64] FLAG: --system-cgroups=""
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685624    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685630    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685633    2575 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685636    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685641    2575 flags.go:64] FLAG: --tls-min-version=""
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685644    2575 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685647    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685650    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 12:54:08.695531 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685653    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685656    2575 flags.go:64] FLAG: --v="2"
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685660    2575 flags.go:64] FLAG: --version="false"
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685664    2575 flags.go:64] FLAG: --vmodule=""
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685668    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.685672    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685806    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685810    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685814    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685818    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685821    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685824    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685828    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685831    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685833    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685836    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685841    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685845    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685848    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685851    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685854    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:54:08.696223 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685859    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685862    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685866    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685869    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685872    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685875    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685878    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685882    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685885    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685888    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685890    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685893    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685896    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685899    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685903    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685907    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685910    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685913    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685916    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685919    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:54:08.697178 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685922    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685925    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685927    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685930    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685933    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685936    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685938    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685943    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685947    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685949    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685952    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685955    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685958    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685961    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685963    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685966    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685969    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685972    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685975    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:54:08.698102 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685978    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685981    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685984    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685987    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685989    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685992    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685995    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.685997    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686016    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686020    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686024    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686029    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686032    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686035    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686038    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686040    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686043    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686046    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686049    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686052    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:54:08.698632 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686056    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686061    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686064    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686069    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686073    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686077    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686080    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686083    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686087    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686089    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686093    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686095    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686098    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686101    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686105    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686108    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686110    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686113    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686116    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:54:08.699372 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686119    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686121    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686124    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686127    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686130    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.686133    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.686827    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.695355    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.695382    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.696290    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.696310    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696414    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696422    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696428    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696433    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:54:08.700354 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696438    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696444    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696449    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696454    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696459    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696463    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696468    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696472    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696477    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696482    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696486    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696490    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696495    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696500    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696504    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696509    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696514    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696519    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696523    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696528    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:54:08.700865 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696533    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696538    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696543    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696548    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696553    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696567    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696572    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696577    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696582    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696587    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696591    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696596    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696600    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696605    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696609    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696614    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696618    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696623    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696628    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696632    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:54:08.701422 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696637    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696641    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696646    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696650    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696655    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696660    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696667    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696674    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696679    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696684    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696690    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696695    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696700    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696705    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696710    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696715    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696719    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696725    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696741    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:54:08.702097 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696746    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696750    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696755    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696760    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696764    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696769    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696773    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696778    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696783    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696788    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696792    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696797    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696802    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696806    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696811    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696815    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696820    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696825    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696829    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696833    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:54:08.702898 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696838    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696842    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696847    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696852    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696857    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696861    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696868    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696874    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696878    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696883    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696888    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696892    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696905    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696911    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696918    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:54:08.703693 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.696924    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.696933    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697213    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697224    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697230    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697236    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697241    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697246    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697252    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697257    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697261    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697266    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697270    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697275    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697279    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:54:08.704144 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697284    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697288    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697293    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697297    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697302    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697307    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697311    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697316    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697321    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697325    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697330    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697335    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697339    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697344    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697349    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697363    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697370    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697377    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697383    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:54:08.704662 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697388    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697396    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697401    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697406    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697411    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697416    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697421    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697425    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697430    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697436    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697443    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697447    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697452    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697457    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697461    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697466    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697470    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697475    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697479    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:54:08.705261 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697484    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697488    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697493    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697498    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697503    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697507    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697512    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697516    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697521    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697526    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697539    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697543    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697548    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697553    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697557    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697561    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697566    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697570    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697575    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:54:08.705761 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697579    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697584    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697588    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697593    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697597    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697602    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697606    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697611    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697615    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697620    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697625    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697629    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697634    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697639    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697643    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697648    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697652    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697656    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697661    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697666    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:54:08.706272 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697670    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697675    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697679    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697683    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697696    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697701    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697706    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697711    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:08.697715    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.697724    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.697941    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.701503    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.704520    2575 server.go:1019] "Starting client certificate rotation"
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.704639    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 12:54:08.706790 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.704685    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 12:54:08.707472 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.707451    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 12:54:08.710350 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.710330    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 12:54:08.723507 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.723484    2575 log.go:25] "Validated CRI v1 runtime API"
Jul 22 12:54:08.725777 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.725764    2575 log.go:25] "Validated CRI v1 image API"
Jul 22 12:54:08.726821 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.726807    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 12:54:08.730745 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.730717    2575 fs.go:135] Filesystem UUIDs: map[2a715818-b833-4ba5-a321-750c526b4ea7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 86ed9220-6bf5-4eb6-9cb2-4e30b10f005e:/dev/nvme0n1p3]
Jul 22 12:54:08.730814 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.730742    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 12:54:08.735909 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.735882    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 12:54:08.737473 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.737364    2575 manager.go:216] Machine: {Timestamp:2026-07-22 12:54:08.735967025 +0000 UTC m=+1.027948053 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101075 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eb5c45b63ae686c78e0d8a3a027bc SystemUUID:ec2eb5c4-5b63-ae68-6c78-e0d8a3a027bc BootID:7c56a09e-b16e-4413-8365-30537cddf858 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bf:d3:a4:62:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bf:d3:a4:62:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:20:b7:6c:bb:45 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 12:54:08.737473 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.737471    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 12:54:08.737627 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.737563    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 12:54:08.741621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.741554    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 12:54:08.741731 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.741589    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-133.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 12:54:08.742732 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.742720    2575 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 12:54:08.742774 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.742735    2575 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 12:54:08.742774 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.742743    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 12:54:08.743413 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.743401    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 12:54:08.744575 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.744564    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 12:54:08.748385 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.748373    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 12:54:08.748500 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.748490    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 12:54:08.748532 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.748522    2575 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 12:54:08.748580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.748540    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 12:54:08.748580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.748561    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 12:54:08.748580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.748574    2575 kubelet.go:398] "Adding apiserver pod source"
Jul 22 12:54:08.748661 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.748587    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 12:54:08.748988 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.748976    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 12:54:08.749040 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.748997    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 12:54:08.751406 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.751374    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 12:54:08.752766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.752753    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 12:54:08.752812 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.752785    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 12:54:08.753404 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753392    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 12:54:08.753446 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753410    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 12:54:08.753446 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753417    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 12:54:08.753446 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753423    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 12:54:08.753446 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753431    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 12:54:08.753446 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753437    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 12:54:08.753579 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753465    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 12:54:08.753579 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753472    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 12:54:08.753579 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753479    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 12:54:08.753579 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753486    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 12:54:08.753579 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753495    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 12:54:08.753579 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.753504    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 12:54:08.754405 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.754395    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 12:54:08.754405 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.754404    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 12:54:08.757703 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.757689    2575 server.go:1268] "Started kubelet"
Jul 22 12:54:08.757843 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.757792    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 12:54:08.757956 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.757863    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 12:54:08.757956 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.757862    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 12:54:08.758720 ip-10-0-140-133 systemd[1]: Started Kubernetes Kubelet.
Jul 22 12:54:08.759278 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.759128    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 12:54:08.761361 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.761343    2575 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 12:54:08.761750 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.761694    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-133.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 12:54:08.761865 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.761780    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 12:54:08.761926 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.761850    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 12:54:08.763289 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.763266    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 12:54:08.765357 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.764543    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-133.ec2.internal.18c49dcb480356ac  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-133.ec2.internal,UID:ip-10-0-140-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-133.ec2.internal,},FirstTimestamp:2026-07-22 12:54:08.757659308 +0000 UTC m=+1.049640336,LastTimestamp:2026-07-22 12:54:08.757659308 +0000 UTC m=+1.049640336,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-133.ec2.internal,}"
Jul 22 12:54:08.765559 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.765542    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ts62z"
Jul 22 12:54:08.766048 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.766034    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 12:54:08.766088 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.766057    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 12:54:08.766197 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.766179    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:08.767509 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.766675    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 12:54:08.767509 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.766831    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 12:54:08.767509 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.766847    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 12:54:08.767509 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.767006    2575 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 12:54:08.767509 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.767015    2575 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 12:54:08.767509 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.767308    2575 factory.go:55] Registering systemd factory
Jul 22 12:54:08.767509 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.767325    2575 factory.go:223] Registration of the systemd container factory successfully
Jul 22 12:54:08.767868 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.767668    2575 factory.go:153] Registering CRI-O factory
Jul 22 12:54:08.767868 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.767687    2575 factory.go:223] Registration of the crio container factory successfully
Jul 22 12:54:08.767868 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.767791    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 12:54:08.767868 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.767849    2575 factory.go:103] Registering Raw factory
Jul 22 12:54:08.768059 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.767878    2575 manager.go:1196] Started watching for new ooms in manager
Jul 22 12:54:08.768947 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.768458    2575 manager.go:349] Starting recovery of all containers
Jul 22 12:54:08.774996 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.774965    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ts62z"
Jul 22 12:54:08.775831 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.775802    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 12:54:08.778235 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.778192    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 12:54:08.782150 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.782133    2575 manager.go:354] Recovery completed
Jul 22 12:54:08.786878 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.786864    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:54:08.789118 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.789097    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:54:08.789202 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.789136    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:54:08.789202 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.789149    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:54:08.789712 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.789698    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 12:54:08.789712 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.789706    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:54:08.789787 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.789710    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 12:54:08.789787 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.789736    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 12:54:08.790457 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.790442    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:54:08.790513 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.790469    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:54:08.790513 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.790479    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:54:08.791444 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.791367    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-133.ec2.internal.18c49dcb49e35b12  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-133.ec2.internal,UID:ip-10-0-140-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-133.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-133.ec2.internal,},FirstTimestamp:2026-07-22 12:54:08.789117714 +0000 UTC m=+1.081098745,LastTimestamp:2026-07-22 12:54:08.789117714 +0000 UTC m=+1.081098745,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-133.ec2.internal,}"
Jul 22 12:54:08.791890 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.791874    2575 policy_none.go:49] "None policy: Start"
Jul 22 12:54:08.791933 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.791901    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 12:54:08.791933 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.791919    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 12:54:08.794097 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.794084    2575 policy_none.go:47] "Start"
Jul 22 12:54:08.829797 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.829606    2575 manager.go:337] "Starting Device Plugin manager"
Jul 22 12:54:08.829944 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.829830    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 12:54:08.829944 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.829846    2575 server.go:87] "Starting device plugin registration server"
Jul 22 12:54:08.830103 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.830090    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 12:54:08.830164 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.830103    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 12:54:08.830240 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.830222    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 12:54:08.830343 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.830329    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 12:54:08.830417 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.830344    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 12:54:08.830838 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.830819    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 12:54:08.830903 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.830868    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:08.932914 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.932882    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:54:08.935063 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.934169    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:54:08.935063 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.934199    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:54:08.935063 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.934210    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:54:08.935063 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.934238    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-133.ec2.internal"
Jul 22 12:54:08.940222 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.940189    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 12:54:08.941550 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.941525    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 12:54:08.941644 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.941563    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 12:54:08.941644 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.941586    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 12:54:08.941644 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.941629    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 12:54:08.943573 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.943548    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-133.ec2.internal"
Jul 22 12:54:08.943659 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.943580    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-133.ec2.internal\": node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:08.943812 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:08.943798    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:54:08.991285 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:08.991259    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.043044 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.042955    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal"]
Jul 22 12:54:09.043154 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.043063    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:54:09.045552 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.045536    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:54:09.045617 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.045567    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:54:09.045617 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.045583    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:54:09.047838 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.047823    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:54:09.047969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.047955    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.048024 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.047985    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:54:09.050709 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.050687    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:54:09.050802 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.050716    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:54:09.050802 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.050721    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:54:09.050802 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.050729    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:54:09.050802 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.050741    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:54:09.050802 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.050752    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:54:09.052936 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.052919    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.053026 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.052945    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:54:09.053764 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.053736    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:54:09.053764 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.053762    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:54:09.053917 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.053777    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:54:09.067393 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.067371    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4b8d72847d0580e7a5d56c90b0538585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal\" (UID: \"4b8d72847d0580e7a5d56c90b0538585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.067471 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.067396    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b8d72847d0580e7a5d56c90b0538585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal\" (UID: \"4b8d72847d0580e7a5d56c90b0538585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.067471 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.067414    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7be2de9b8daaa41c577843a39b0d55eb-config\") pod \"kube-apiserver-proxy-ip-10-0-140-133.ec2.internal\" (UID: \"7be2de9b8daaa41c577843a39b0d55eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.076896 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.076878    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-133.ec2.internal\" not found" node="ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.081663 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.081648    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-133.ec2.internal\" not found" node="ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.091789 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.091770    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.167799 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.167768    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4b8d72847d0580e7a5d56c90b0538585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal\" (UID: \"4b8d72847d0580e7a5d56c90b0538585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.167911 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.167835    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4b8d72847d0580e7a5d56c90b0538585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal\" (UID: \"4b8d72847d0580e7a5d56c90b0538585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.167911 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.167876    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b8d72847d0580e7a5d56c90b0538585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal\" (UID: \"4b8d72847d0580e7a5d56c90b0538585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.168001 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.167915    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7be2de9b8daaa41c577843a39b0d55eb-config\") pod \"kube-apiserver-proxy-ip-10-0-140-133.ec2.internal\" (UID: \"7be2de9b8daaa41c577843a39b0d55eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.168001 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.167978    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7be2de9b8daaa41c577843a39b0d55eb-config\") pod \"kube-apiserver-proxy-ip-10-0-140-133.ec2.internal\" (UID: \"7be2de9b8daaa41c577843a39b0d55eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.168001 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.167979    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b8d72847d0580e7a5d56c90b0538585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal\" (UID: \"4b8d72847d0580e7a5d56c90b0538585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.192225 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.192198    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.293469 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.293388    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.379258 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.379220    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.385351 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.385329    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:09.394584 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.394566    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.495750 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.495711    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.597039 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.596999    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.698342 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.698309    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.702811 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.702795    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 12:54:09.702982 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.702964    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 12:54:09.781604 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.781555    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 12:49:09 +0000 UTC" deadline="2028-03-03 22:06:41.709930081 +0000 UTC"
Jul 22 12:54:09.781604 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.781593    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14169h12m31.928340322s"
Jul 22 12:54:09.783373 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.783354    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:54:09.798923 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.798892    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.898153 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.898110    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:54:09.899062 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.899044    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:09.983972 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:09.983909    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be2de9b8daaa41c577843a39b0d55eb.slice/crio-e0f5fbb6a7a5fe359dd6dd5ab9b1731c1ca72859af2392541dd20f8a2e09a674 WatchSource:0}: Error finding container e0f5fbb6a7a5fe359dd6dd5ab9b1731c1ca72859af2392541dd20f8a2e09a674: Status 404 returned error can't find the container with id e0f5fbb6a7a5fe359dd6dd5ab9b1731c1ca72859af2392541dd20f8a2e09a674
Jul 22 12:54:09.994908 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:09.994254    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:54:09.999904 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:09.999859    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:10.101514 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.101460    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-133.ec2.internal\" not found"
Jul 22 12:54:10.159389 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.159318    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:54:10.166712 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.166691    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:10.178947 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.178906    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 12:54:10.180057 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.180042    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal"
Jul 22 12:54:10.187208 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.187194    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 12:54:10.750389 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.750346    2575 apiserver.go:52] "Watching apiserver"
Jul 22 12:54:10.760760 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.759688    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 12:54:10.760760 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.760286    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-pz4kt","openshift-multus/network-metrics-daemon-mncs6","openshift-network-diagnostics/network-check-target-v25hs","openshift-network-operator/iptables-alerter-d9gvw","kube-system/global-pull-secret-syncer-5cxpk","kube-system/konnectivity-agent-khrvp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc","openshift-cluster-node-tuning-operator/tuned-9bgt8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal","openshift-multus/multus-additional-cni-plugins-wjtlj","openshift-ovn-kubernetes/ovnkube-node-z8g9v","kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal","openshift-dns/node-resolver-qsx49","openshift-image-registry/node-ca-nsbth"]
Jul 22 12:54:10.763796 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.763064    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.766265 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.766026    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 12:54:10.766265 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.766122    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 12:54:10.766460 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.766300    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 12:54:10.768452 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.767025    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.768452 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.767562    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 12:54:10.772195 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.769253    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5bjlv\""
Jul 22 12:54:10.772195 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.769662    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-68tlv\""
Jul 22 12:54:10.772195 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.769813    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 12:54:10.772195 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.770040    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 12:54:10.773817 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.773624    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:10.773817 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.773692    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:10.776177 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.776151    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:10.777713 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.777690    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.777832 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.777732    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-registration-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.777832 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.777760    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-device-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.777832 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.777794    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.778025 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.777842    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkhvg\" (UniqueName: \"kubernetes.io/projected/bfb3c025-7b4e-45c8-aced-0602196c44f4-kube-api-access-qkhvg\") pod \"node-resolver-qsx49\" (UID: \"bfb3c025-7b4e-45c8-aced-0602196c44f4\") " pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.778025 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.777911    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-socket-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.778025 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.777953    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-sys-fs\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.778025 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.778002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txhx7\" (UniqueName: \"kubernetes.io/projected/fac2e041-e652-495f-aaad-b007dfcc11d1-kube-api-access-txhx7\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.778236 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.778111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bfb3c025-7b4e-45c8-aced-0602196c44f4-hosts-file\") pod \"node-resolver-qsx49\" (UID: \"bfb3c025-7b4e-45c8-aced-0602196c44f4\") " pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.778236 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.778141    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfb3c025-7b4e-45c8-aced-0602196c44f4-tmp-dir\") pod \"node-resolver-qsx49\" (UID: \"bfb3c025-7b4e-45c8-aced-0602196c44f4\") " pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.778236 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.778165    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:10.778404 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.778357    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:10.778459 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.778421    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:10.779063 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.779044    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 12:54:10.779165 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.779116    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:54:10.779506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.779318    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 12:54:10.780035 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.779970    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4gk7r\""
Jul 22 12:54:10.781250 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.780612    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:10.783059 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.782806    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 12:54:10.783208 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.783189    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:10.784049 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.783856    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-h29fb\""
Jul 22 12:54:10.784049 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.784016    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 12:54:10.784049 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.784016    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 12:54:10.785296 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.785264    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.785701 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.785677    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 12:54:10.789177 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.787196    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 12:54:10.789177 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.787637    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 12:54:10.789177 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.787861    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:10.789177 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.787939    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:10.789177 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.788670    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 12:54:10.790749 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.790717    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9tcrr\""
Jul 22 12:54:10.791294 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.791273    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 12:54:10.791426 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.791402    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qjg7b\""
Jul 22 12:54:10.791498 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.791439    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 12:54:10.791697 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.791682    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 12:54:10.792932 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.792269    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.794837 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.794642    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 12:54:10.795496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.795476    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.796191 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.795784    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 12:54:10.796525 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.796362    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-htm7z\""
Jul 22 12:54:10.798026 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.797965    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.798142 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.797950    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 12:54:10.798206 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.798172    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 12:54:10.799487 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.799467    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k8f69\""
Jul 22 12:54:10.799736 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.799720    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 12:54:10.799963 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.799947    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 12:54:10.800367 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.800341    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 12:54:10.800582 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.800565    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 12:54:10.801060 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.801040    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:54:10.801150 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.801058    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gb9p5\""
Jul 22 12:54:10.801328 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.801294    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 12:54:10.806496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.806466    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jqcqf"
Jul 22 12:54:10.817018 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.816602    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jqcqf"
Jul 22 12:54:10.868078 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.867991    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 12:54:10.878532 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878498    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-run-netns\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.878646 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878540    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-daemon-config\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.878646 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878569    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-os-release\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.878646 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878595    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2436034c-df7c-4212-b9d8-b85737f6c858-dbus\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:10.878646 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878618    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-systemd-units\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.878646 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878642    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-slash\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878668    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-run-openvswitch\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878696    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-cnibin\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-socket-dir-parent\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878745    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-hostroot\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878768    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-var-lib-kubelet\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878794    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8946923f-734c-497c-a2ee-73f244a994ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878845    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-run-ovn\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878871    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.878916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878908    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878933    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffb63a5d-e35a-4530-8b14-d95314b01700-tmp\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.878968    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/762bf3f8-33f0-4a22-93bc-18ff543c554e-iptables-alerter-script\") pod \"iptables-alerter-d9gvw\" (UID: \"762bf3f8-33f0-4a22-93bc-18ff543c554e\") " pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879011    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055-host\") pod \"node-ca-nsbth\" (UID: \"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055\") " pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879037    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxkcp\" (UniqueName: \"kubernetes.io/projected/05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055-kube-api-access-cxkcp\") pod \"node-ca-nsbth\" (UID: \"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055\") " pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879061    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-var-lib-openvswitch\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879096    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-socket-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879124    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91358d07-1210-40d9-a108-654aae4213f6-agent-certs\") pod \"konnectivity-agent-khrvp\" (UID: \"91358d07-1210-40d9-a108-654aae4213f6\") " pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879185    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91358d07-1210-40d9-a108-654aae4213f6-konnectivity-ca\") pod \"konnectivity-agent-khrvp\" (UID: \"91358d07-1210-40d9-a108-654aae4213f6\") " pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879214    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-var-lib-cni-bin\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879241    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-kubernetes\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879267    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-sysctl-d\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879292    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-lib-modules\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879321    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-sys-fs\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879348    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879376    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-cnibin\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879406    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-sysconfig\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.879496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879450    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-sys\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879527    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-sys-fs\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-run-systemd\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879623    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-socket-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879621    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-node-log\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-env-overrides\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-device-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879772    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-system-cni-dir\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879801    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-device-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-var-lib-cni-multus\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879885    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-systemd\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879920    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbbdb\" (UniqueName: \"kubernetes.io/projected/8946923f-734c-497c-a2ee-73f244a994ee-kube-api-access-vbbdb\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.879953    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txhx7\" (UniqueName: \"kubernetes.io/projected/fac2e041-e652-495f-aaad-b007dfcc11d1-kube-api-access-txhx7\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bfb3c025-7b4e-45c8-aced-0602196c44f4-hosts-file\") pod \"node-resolver-qsx49\" (UID: \"bfb3c025-7b4e-45c8-aced-0602196c44f4\") " pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880050    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-cni-binary-copy\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880111    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bfb3c025-7b4e-45c8-aced-0602196c44f4-hosts-file\") pod \"node-resolver-qsx49\" (UID: \"bfb3c025-7b4e-45c8-aced-0602196c44f4\") " pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880115    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-run-multus-certs\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.880392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880163    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-etc-kubernetes\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-log-socket\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880246    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880285    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8946923f-734c-497c-a2ee-73f244a994ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880321    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-tuned\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880357    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgzwz\" (UniqueName: \"kubernetes.io/projected/ffb63a5d-e35a-4530-8b14-d95314b01700-kube-api-access-mgzwz\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5ccb\" (UniqueName: \"kubernetes.io/projected/762bf3f8-33f0-4a22-93bc-18ff543c554e-kube-api-access-k5ccb\") pod \"iptables-alerter-d9gvw\" (UID: \"762bf3f8-33f0-4a22-93bc-18ff543c554e\") " pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880433    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055-serviceca\") pod \"node-ca-nsbth\" (UID: \"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055\") " pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880466    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880499    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-run-netns\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880525    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfb3c025-7b4e-45c8-aced-0602196c44f4-tmp-dir\") pod \"node-resolver-qsx49\" (UID: \"bfb3c025-7b4e-45c8-aced-0602196c44f4\") " pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880551    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-run-k8s-cni-cncf-io\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880576    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-var-lib-kubelet\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880616    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9bj5\" (UniqueName: \"kubernetes.io/projected/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-kube-api-access-b9bj5\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880638    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-run\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-host\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.881359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/762bf3f8-33f0-4a22-93bc-18ff543c554e-host-slash\") pod \"iptables-alerter-d9gvw\" (UID: \"762bf3f8-33f0-4a22-93bc-18ff543c554e\") " pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880712    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-ovnkube-script-lib\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880764    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-registration-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880788    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkhvg\" (UniqueName: \"kubernetes.io/projected/bfb3c025-7b4e-45c8-aced-0602196c44f4-kube-api-access-qkhvg\") pod \"node-resolver-qsx49\" (UID: \"bfb3c025-7b4e-45c8-aced-0602196c44f4\") " pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880837    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-cni-dir\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-os-release\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8946923f-734c-497c-a2ee-73f244a994ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880935    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-sysctl-conf\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.880961    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-etc-openvswitch\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881010    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8blp\" (UniqueName: \"kubernetes.io/projected/5b615de2-b59e-417c-95d5-42cb2c7fd256-kube-api-access-v8blp\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881037    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-kubelet\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881063    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-cni-netd\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-ovnkube-config\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881125    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-ovn-node-metrics-cert\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881154    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-system-cni-dir\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.882218 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881180    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2436034c-df7c-4212-b9d8-b85737f6c858-kubelet-config\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:10.883036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rngcb\" (UniqueName: \"kubernetes.io/projected/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-kube-api-access-rngcb\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.883036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881238    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-cni-bin\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.883036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881268    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.883036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881374    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-conf-dir\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.883036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881402    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-modprobe-d\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.883036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.881565    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.883036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.882115    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfb3c025-7b4e-45c8-aced-0602196c44f4-tmp-dir\") pod \"node-resolver-qsx49\" (UID: \"bfb3c025-7b4e-45c8-aced-0602196c44f4\") " pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.883036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.882211    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.883036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.882261    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fac2e041-e652-495f-aaad-b007dfcc11d1-registration-dir\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.889357 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.889318    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 12:54:10.889797 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.889773    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:54:10.889909 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.889808    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:54:10.889909 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.889822    2575 projected.go:196] Error preparing data for projected volume kube-api-access-66qtv for pod openshift-network-diagnostics/network-check-target-v25hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:10.890037 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.889910    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv podName:e1b75105-2d2e-486a-b65d-2e0242c535b9 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:11.389876054 +0000 UTC m=+3.681857083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-66qtv" (UniqueName: "kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv") pod "network-check-target-v25hs" (UID: "e1b75105-2d2e-486a-b65d-2e0242c535b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:10.894905 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.894826    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txhx7\" (UniqueName: \"kubernetes.io/projected/fac2e041-e652-495f-aaad-b007dfcc11d1-kube-api-access-txhx7\") pod \"aws-ebs-csi-driver-node-bkwgc\" (UID: \"fac2e041-e652-495f-aaad-b007dfcc11d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:10.894905 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.894842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkhvg\" (UniqueName: \"kubernetes.io/projected/bfb3c025-7b4e-45c8-aced-0602196c44f4-kube-api-access-qkhvg\") pod \"node-resolver-qsx49\" (UID: \"bfb3c025-7b4e-45c8-aced-0602196c44f4\") " pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:10.949132 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.949044    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal" event={"ID":"4b8d72847d0580e7a5d56c90b0538585","Type":"ContainerStarted","Data":"50f30b60b9ab50297e4ae684f5d50654f27d4e25b01cf2097df5df082f45efe3"}
Jul 22 12:54:10.950645 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.950588    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal" event={"ID":"7be2de9b8daaa41c577843a39b0d55eb","Type":"ContainerStarted","Data":"e0f5fbb6a7a5fe359dd6dd5ab9b1731c1ca72859af2392541dd20f8a2e09a674"}
Jul 22 12:54:10.981968 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.981878    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-cni-dir\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.981968 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.981915    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-os-release\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.981968 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.981934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8946923f-734c-497c-a2ee-73f244a994ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.981976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-sysctl-conf\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.981997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-etc-openvswitch\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982018    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8blp\" (UniqueName: \"kubernetes.io/projected/5b615de2-b59e-417c-95d5-42cb2c7fd256-kube-api-access-v8blp\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-kubelet\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982058    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-cni-netd\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982077    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-ovnkube-config\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982098    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-ovn-node-metrics-cert\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-system-cni-dir\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982141    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2436034c-df7c-4212-b9d8-b85737f6c858-kubelet-config\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982164    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rngcb\" (UniqueName: \"kubernetes.io/projected/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-kube-api-access-rngcb\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982182    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-cni-dir\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982188    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-cni-bin\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982239    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-conf-dir\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-cni-bin\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982266    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-modprobe-d\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982293    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-run-netns\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982310    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-os-release\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982319    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-daemon-config\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982346    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-os-release\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982369    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2436034c-df7c-4212-b9d8-b85737f6c858-dbus\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982392    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-systemd-units\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982430    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-slash\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982455    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-run-openvswitch\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982465    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-cni-netd\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982479    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-etc-openvswitch\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982498    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-system-cni-dir\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982525    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-conf-dir\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.982622 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982548    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-cnibin\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982631    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-sysctl-conf\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982652    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-modprobe-d\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-run-netns\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982716    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2436034c-df7c-4212-b9d8-b85737f6c858-dbus\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982765    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-systemd-units\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982781    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-os-release\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-cnibin\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982817    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-slash\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982827    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-socket-dir-parent\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982858    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-run-openvswitch\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982867    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-hostroot\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982888    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8946923f-734c-497c-a2ee-73f244a994ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982908    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-hostroot\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-var-lib-kubelet\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982953    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.982962    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-socket-dir-parent\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983004    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8946923f-734c-497c-a2ee-73f244a994ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.983506 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983029    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-var-lib-kubelet\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983030    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-run-ovn\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983072    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-run-ovn\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983070    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983108    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983115    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983142    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffb63a5d-e35a-4530-8b14-d95314b01700-tmp\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-kubelet\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-ovnkube-config\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.983225    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983278    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/762bf3f8-33f0-4a22-93bc-18ff543c554e-iptables-alerter-script\") pod \"iptables-alerter-d9gvw\" (UID: \"762bf3f8-33f0-4a22-93bc-18ff543c554e\") " pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.983292    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret podName:2436034c-df7c-4212-b9d8-b85737f6c858 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:11.483273901 +0000 UTC m=+3.775254917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret") pod "global-pull-secret-syncer-5cxpk" (UID: "2436034c-df7c-4212-b9d8-b85737f6c858") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983319    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055-host\") pod \"node-ca-nsbth\" (UID: \"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055\") " pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983346    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxkcp\" (UniqueName: \"kubernetes.io/projected/05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055-kube-api-access-cxkcp\") pod \"node-ca-nsbth\" (UID: \"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055\") " pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983364    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-var-lib-openvswitch\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983399    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-multus-daemon-config\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.984440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91358d07-1210-40d9-a108-654aae4213f6-agent-certs\") pod \"konnectivity-agent-khrvp\" (UID: \"91358d07-1210-40d9-a108-654aae4213f6\") " pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983229    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2436034c-df7c-4212-b9d8-b85737f6c858-kubelet-config\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983436    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91358d07-1210-40d9-a108-654aae4213f6-konnectivity-ca\") pod \"konnectivity-agent-khrvp\" (UID: \"91358d07-1210-40d9-a108-654aae4213f6\") " pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983473    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-var-lib-cni-bin\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983501    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-kubernetes\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983524    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-sysctl-d\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-lib-modules\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-cnibin\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-sysconfig\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983617    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055-host\") pod \"node-ca-nsbth\" (UID: \"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055\") " pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983640    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-sys\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983666    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-run-systemd\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983692    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-node-log\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983710    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8946923f-734c-497c-a2ee-73f244a994ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-env-overrides\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983743    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-system-cni-dir\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983769    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-var-lib-cni-multus\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983773    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8946923f-734c-497c-a2ee-73f244a994ee-cnibin\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.985227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983793    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-systemd\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbbdb\" (UniqueName: \"kubernetes.io/projected/8946923f-734c-497c-a2ee-73f244a994ee-kube-api-access-vbbdb\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-cni-binary-copy\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983868    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/762bf3f8-33f0-4a22-93bc-18ff543c554e-iptables-alerter-script\") pod \"iptables-alerter-d9gvw\" (UID: \"762bf3f8-33f0-4a22-93bc-18ff543c554e\") " pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983878    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-run-multus-certs\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-etc-kubernetes\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983918    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-sysctl-d\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983932    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-lib-modules\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983940    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-system-cni-dir\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983824    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-var-lib-cni-bin\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983952    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91358d07-1210-40d9-a108-654aae4213f6-konnectivity-ca\") pod \"konnectivity-agent-khrvp\" (UID: \"91358d07-1210-40d9-a108-654aae4213f6\") " pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983930    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-log-socket\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983981    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-kubernetes\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984009    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-sysconfig\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983994    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-var-lib-cni-multus\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983410    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-var-lib-openvswitch\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984045    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-log-socket\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.983998    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.986140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984073    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-sys\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-node-log\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984089    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8946923f-734c-497c-a2ee-73f244a994ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984127    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-run-multus-certs\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984209    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984447    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-env-overrides\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-etc-kubernetes\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984492    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-tuned\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984502    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-systemd\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984523    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgzwz\" (UniqueName: \"kubernetes.io/projected/ffb63a5d-e35a-4530-8b14-d95314b01700-kube-api-access-mgzwz\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984533    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-cni-binary-copy\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984552    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5ccb\" (UniqueName: \"kubernetes.io/projected/762bf3f8-33f0-4a22-93bc-18ff543c554e-kube-api-access-k5ccb\") pod \"iptables-alerter-d9gvw\" (UID: \"762bf3f8-33f0-4a22-93bc-18ff543c554e\") " pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984579    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055-serviceca\") pod \"node-ca-nsbth\" (UID: \"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055\") " pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984583    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-run-systemd\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984583    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8946923f-734c-497c-a2ee-73f244a994ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984642    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-run-netns\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.987030 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984667    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-run-k8s-cni-cncf-io\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984691    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-var-lib-kubelet\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984715    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9bj5\" (UniqueName: \"kubernetes.io/projected/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-kube-api-access-b9bj5\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984726    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-host-run-netns\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-run\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984761    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-host\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984788    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/762bf3f8-33f0-4a22-93bc-18ff543c554e-host-slash\") pod \"iptables-alerter-d9gvw\" (UID: \"762bf3f8-33f0-4a22-93bc-18ff543c554e\") " pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.984808    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-ovnkube-script-lib\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-run-k8s-cni-cncf-io\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:10.984868    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs podName:5b615de2-b59e-417c-95d5-42cb2c7fd256 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:11.484851077 +0000 UTC m=+3.776832107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs") pod "network-metrics-daemon-mncs6" (UID: "5b615de2-b59e-417c-95d5-42cb2c7fd256") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984887    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-host-var-lib-kubelet\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984927    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-host\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.984941    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ffb63a5d-e35a-4530-8b14-d95314b01700-run\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.985000    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/762bf3f8-33f0-4a22-93bc-18ff543c554e-host-slash\") pod \"iptables-alerter-d9gvw\" (UID: \"762bf3f8-33f0-4a22-93bc-18ff543c554e\") " pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.985154    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055-serviceca\") pod \"node-ca-nsbth\" (UID: \"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055\") " pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.985379    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-ovnkube-script-lib\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.987876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.987838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ffb63a5d-e35a-4530-8b14-d95314b01700-etc-tuned\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:10.988822 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.987974    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91358d07-1210-40d9-a108-654aae4213f6-agent-certs\") pod \"konnectivity-agent-khrvp\" (UID: \"91358d07-1210-40d9-a108-654aae4213f6\") " pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:10.988938 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.988913    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-ovn-node-metrics-cert\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:10.990527 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:10.990503    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffb63a5d-e35a-4530-8b14-d95314b01700-tmp\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:11.003042 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.002897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8blp\" (UniqueName: \"kubernetes.io/projected/5b615de2-b59e-417c-95d5-42cb2c7fd256-kube-api-access-v8blp\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:11.004176 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.004128    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxkcp\" (UniqueName: \"kubernetes.io/projected/05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055-kube-api-access-cxkcp\") pod \"node-ca-nsbth\" (UID: \"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055\") " pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:11.006434 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.006403    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgzwz\" (UniqueName: \"kubernetes.io/projected/ffb63a5d-e35a-4530-8b14-d95314b01700-kube-api-access-mgzwz\") pod \"tuned-9bgt8\" (UID: \"ffb63a5d-e35a-4530-8b14-d95314b01700\") " pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:11.007237 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.007212    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbbdb\" (UniqueName: \"kubernetes.io/projected/8946923f-734c-497c-a2ee-73f244a994ee-kube-api-access-vbbdb\") pod \"multus-additional-cni-plugins-wjtlj\" (UID: \"8946923f-734c-497c-a2ee-73f244a994ee\") " pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:11.007343 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.007243    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rngcb\" (UniqueName: \"kubernetes.io/projected/0a9ea2b6-2062-4f0a-aa47-63859b3b2efd-kube-api-access-rngcb\") pod \"ovnkube-node-z8g9v\" (UID: \"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:11.008446 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.008391    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5ccb\" (UniqueName: \"kubernetes.io/projected/762bf3f8-33f0-4a22-93bc-18ff543c554e-kube-api-access-k5ccb\") pod \"iptables-alerter-d9gvw\" (UID: \"762bf3f8-33f0-4a22-93bc-18ff543c554e\") " pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:11.008910 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.008887    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9bj5\" (UniqueName: \"kubernetes.io/projected/0ba7a0b4-cc88-4042-beb2-3c6be9e63a20-kube-api-access-b9bj5\") pod \"multus-pz4kt\" (UID: \"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20\") " pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:11.085541 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.085502    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc"
Jul 22 12:54:11.094760 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.094625    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qsx49"
Jul 22 12:54:11.109518 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.109491    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d9gvw"
Jul 22 12:54:11.116677 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.116652    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:11.126670 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.126644    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nsbth"
Jul 22 12:54:11.130509 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.130491    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:54:11.142791 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.142763    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pz4kt"
Jul 22 12:54:11.151785 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.151764    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wjtlj"
Jul 22 12:54:11.163049 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.163023    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:11.168142 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.168119    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9bgt8"
Jul 22 12:54:11.488996 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.488958    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:11.489160 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.489006    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:11.489160 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.489038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:11.489160 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.489141    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:11.489321 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.489171    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:54:11.489321 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.489193    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:11.489321 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.489208    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs podName:5b615de2-b59e-417c-95d5-42cb2c7fd256 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:12.489191544 +0000 UTC m=+4.781172569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs") pod "network-metrics-daemon-mncs6" (UID: "5b615de2-b59e-417c-95d5-42cb2c7fd256") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:11.489321 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.489261    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret podName:2436034c-df7c-4212-b9d8-b85737f6c858 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:12.489243582 +0000 UTC m=+4.781224625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret") pod "global-pull-secret-syncer-5cxpk" (UID: "2436034c-df7c-4212-b9d8-b85737f6c858") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:11.489321 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.489198    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:54:11.489321 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.489283    2575 projected.go:196] Error preparing data for projected volume kube-api-access-66qtv for pod openshift-network-diagnostics/network-check-target-v25hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:11.489321 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.489322    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv podName:e1b75105-2d2e-486a-b65d-2e0242c535b9 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:12.489308794 +0000 UTC m=+4.781289816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-66qtv" (UniqueName: "kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv") pod "network-check-target-v25hs" (UID: "e1b75105-2d2e-486a-b65d-2e0242c535b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:11.797840 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:11.797764    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac2e041_e652_495f_aaad_b007dfcc11d1.slice/crio-3c246691f1f3cf2e47cdbfc9bcdb9e6c703b19da1c54339450d9f07839cc55ed WatchSource:0}: Error finding container 3c246691f1f3cf2e47cdbfc9bcdb9e6c703b19da1c54339450d9f07839cc55ed: Status 404 returned error can't find the container with id 3c246691f1f3cf2e47cdbfc9bcdb9e6c703b19da1c54339450d9f07839cc55ed
Jul 22 12:54:11.819861 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.819824    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 12:49:11 +0000 UTC" deadline="2028-04-26 09:31:40.57888326 +0000 UTC"
Jul 22 12:54:11.819861 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.819852    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15452h37m28.759035909s"
Jul 22 12:54:11.942502 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.942468    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:11.942662 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.942608    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:11.942738 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.942670    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:11.942835 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:11.942806    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:11.953372 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:11.953344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc" event={"ID":"fac2e041-e652-495f-aaad-b007dfcc11d1","Type":"ContainerStarted","Data":"3c246691f1f3cf2e47cdbfc9bcdb9e6c703b19da1c54339450d9f07839cc55ed"}
Jul 22 12:54:12.175186 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:12.175153    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762bf3f8_33f0_4a22_93bc_18ff543c554e.slice/crio-51ce150779088f8effcccdd885d0e9152c5b4e3513bd3048dd9cb025f9b6cfff WatchSource:0}: Error finding container 51ce150779088f8effcccdd885d0e9152c5b4e3513bd3048dd9cb025f9b6cfff: Status 404 returned error can't find the container with id 51ce150779088f8effcccdd885d0e9152c5b4e3513bd3048dd9cb025f9b6cfff
Jul 22 12:54:12.176389 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:12.176360    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91358d07_1210_40d9_a108_654aae4213f6.slice/crio-9c3492c5bc3c05e7a96edf116ece7717eac4c7ae8fa0c5bc19336304089c27f4 WatchSource:0}: Error finding container 9c3492c5bc3c05e7a96edf116ece7717eac4c7ae8fa0c5bc19336304089c27f4: Status 404 returned error can't find the container with id 9c3492c5bc3c05e7a96edf116ece7717eac4c7ae8fa0c5bc19336304089c27f4
Jul 22 12:54:12.180086 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:12.178874    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb63a5d_e35a_4530_8b14_d95314b01700.slice/crio-878f2cba5bab4f9af1ff66901bd2daf51b9dc8f1a667ada75b13cb7d2e867ce0 WatchSource:0}: Error finding container 878f2cba5bab4f9af1ff66901bd2daf51b9dc8f1a667ada75b13cb7d2e867ce0: Status 404 returned error can't find the container with id 878f2cba5bab4f9af1ff66901bd2daf51b9dc8f1a667ada75b13cb7d2e867ce0
Jul 22 12:54:12.187099 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:12.186846    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ba7a0b4_cc88_4042_beb2_3c6be9e63a20.slice/crio-7ee3520198d8c901777de22b1fb86291773aba57a0b0beafc74386cb9d043adc WatchSource:0}: Error finding container 7ee3520198d8c901777de22b1fb86291773aba57a0b0beafc74386cb9d043adc: Status 404 returned error can't find the container with id 7ee3520198d8c901777de22b1fb86291773aba57a0b0beafc74386cb9d043adc
Jul 22 12:54:12.187208 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:12.187125    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8946923f_734c_497c_a2ee_73f244a994ee.slice/crio-391bb5bea617d2ca477ec9cdc411050675098f6cc6035a34212b91cbbd09237c WatchSource:0}: Error finding container 391bb5bea617d2ca477ec9cdc411050675098f6cc6035a34212b91cbbd09237c: Status 404 returned error can't find the container with id 391bb5bea617d2ca477ec9cdc411050675098f6cc6035a34212b91cbbd09237c
Jul 22 12:54:12.210233 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:12.210202    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05bffc4a_a65c_41b9_9bd9_7d5bf5f5d055.slice/crio-9d1b4b6f43244119375fe50ab90d2295cd189f58a69472a40dcad2d827446c61 WatchSource:0}: Error finding container 9d1b4b6f43244119375fe50ab90d2295cd189f58a69472a40dcad2d827446c61: Status 404 returned error can't find the container with id 9d1b4b6f43244119375fe50ab90d2295cd189f58a69472a40dcad2d827446c61
Jul 22 12:54:12.211076 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:12.211049    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb3c025_7b4e_45c8_aced_0602196c44f4.slice/crio-7c56f738fd09d988f6fac731c09b689e4f3bf91f6498669c17b656245fba07ca WatchSource:0}: Error finding container 7c56f738fd09d988f6fac731c09b689e4f3bf91f6498669c17b656245fba07ca: Status 404 returned error can't find the container with id 7c56f738fd09d988f6fac731c09b689e4f3bf91f6498669c17b656245fba07ca
Jul 22 12:54:12.497653 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:12.497411    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:12.497849 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:12.497693    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:12.497849 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:12.497602    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:54:12.497849 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:12.497746    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:54:12.497849 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:12.497754    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:12.497849 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:12.497760    2575 projected.go:196] Error preparing data for projected volume kube-api-access-66qtv for pod openshift-network-diagnostics/network-check-target-v25hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:12.497849 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:12.497830    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:12.498104 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:12.497867    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv podName:e1b75105-2d2e-486a-b65d-2e0242c535b9 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:14.497844211 +0000 UTC m=+6.789825240 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-66qtv" (UniqueName: "kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv") pod "network-check-target-v25hs" (UID: "e1b75105-2d2e-486a-b65d-2e0242c535b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:12.498104 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:12.497888    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs podName:5b615de2-b59e-417c-95d5-42cb2c7fd256 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:14.497877174 +0000 UTC m=+6.789858192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs") pod "network-metrics-daemon-mncs6" (UID: "5b615de2-b59e-417c-95d5-42cb2c7fd256") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:12.498104 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:12.497906    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:12.498104 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:12.497965    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret podName:2436034c-df7c-4212-b9d8-b85737f6c858 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:14.497950897 +0000 UTC m=+6.789931914 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret") pod "global-pull-secret-syncer-5cxpk" (UID: "2436034c-df7c-4212-b9d8-b85737f6c858") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:12.821128 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:12.821029    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 12:49:11 +0000 UTC" deadline="2028-04-24 07:27:51.3607582 +0000 UTC"
Jul 22 12:54:12.821128 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:12.821093    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15402h33m38.539669976s"
Jul 22 12:54:12.952910 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:12.952392    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:12.952910 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:12.952534    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:12.971098 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:12.970936    2575 generic.go:356] "Generic (PLEG): container finished" podID="4b8d72847d0580e7a5d56c90b0538585" containerID="d467ed1e7eafc568694260bd2257db98e9677ede36b2f74b46899413782bb9e7" exitCode=0
Jul 22 12:54:12.971504 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:12.971391    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal" event={"ID":"4b8d72847d0580e7a5d56c90b0538585","Type":"ContainerDied","Data":"d467ed1e7eafc568694260bd2257db98e9677ede36b2f74b46899413782bb9e7"}
Jul 22 12:54:12.983270 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:12.983237    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pz4kt" event={"ID":"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20","Type":"ContainerStarted","Data":"7ee3520198d8c901777de22b1fb86291773aba57a0b0beafc74386cb9d043adc"}
Jul 22 12:54:13.004786 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.004413    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerStarted","Data":"e244af3ae80e34c5e3e917a46a3af4f0275bc06ca5b0d0f1e160bc8419560fd0"}
Jul 22 12:54:13.021372 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.021320    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d9gvw" event={"ID":"762bf3f8-33f0-4a22-93bc-18ff543c554e","Type":"ContainerStarted","Data":"51ce150779088f8effcccdd885d0e9152c5b4e3513bd3048dd9cb025f9b6cfff"}
Jul 22 12:54:13.039448 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.039026    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal" event={"ID":"7be2de9b8daaa41c577843a39b0d55eb","Type":"ContainerStarted","Data":"bfed969aa93f3fe8cbfd4f761fcd9170433bd6b4bdbe97647cc9de79151f7cf4"}
Jul 22 12:54:13.049173 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.049137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qsx49" event={"ID":"bfb3c025-7b4e-45c8-aced-0602196c44f4","Type":"ContainerStarted","Data":"7c56f738fd09d988f6fac731c09b689e4f3bf91f6498669c17b656245fba07ca"}
Jul 22 12:54:13.062147 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.062098    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nsbth" event={"ID":"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055","Type":"ContainerStarted","Data":"9d1b4b6f43244119375fe50ab90d2295cd189f58a69472a40dcad2d827446c61"}
Jul 22 12:54:13.073287 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.073214    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjtlj" event={"ID":"8946923f-734c-497c-a2ee-73f244a994ee","Type":"ContainerStarted","Data":"391bb5bea617d2ca477ec9cdc411050675098f6cc6035a34212b91cbbd09237c"}
Jul 22 12:54:13.086836 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.086779    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9bgt8" event={"ID":"ffb63a5d-e35a-4530-8b14-d95314b01700","Type":"ContainerStarted","Data":"878f2cba5bab4f9af1ff66901bd2daf51b9dc8f1a667ada75b13cb7d2e867ce0"}
Jul 22 12:54:13.106706 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.106668    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-khrvp" event={"ID":"91358d07-1210-40d9-a108-654aae4213f6","Type":"ContainerStarted","Data":"9c3492c5bc3c05e7a96edf116ece7717eac4c7ae8fa0c5bc19336304089c27f4"}
Jul 22 12:54:13.942584 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.942421    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:13.942584 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:13.942548    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:13.943167 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:13.942915    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:13.943167 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:13.943024    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:14.139035 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:14.138995    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal" event={"ID":"4b8d72847d0580e7a5d56c90b0538585","Type":"ContainerStarted","Data":"a77ad530b41c4692ed639d1aad86ec04840b25c659e8c522b4198aec6b469a06"}
Jul 22 12:54:14.155390 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:14.154621    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-133.ec2.internal" podStartSLOduration=4.154601027 podStartE2EDuration="4.154601027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:14.154201327 +0000 UTC m=+6.446182361" watchObservedRunningTime="2026-07-22 12:54:14.154601027 +0000 UTC m=+6.446582067"
Jul 22 12:54:14.155390 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:14.155042    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-133.ec2.internal" podStartSLOduration=4.155033215 podStartE2EDuration="4.155033215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:13.056961024 +0000 UTC m=+5.348942063" watchObservedRunningTime="2026-07-22 12:54:14.155033215 +0000 UTC m=+6.447014250"
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:14.519440    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:14.519512    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:14.519551    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:14.519687    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:14.519704    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:14.519717    2575 projected.go:196] Error preparing data for projected volume kube-api-access-66qtv for pod openshift-network-diagnostics/network-check-target-v25hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:14.519773    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv podName:e1b75105-2d2e-486a-b65d-2e0242c535b9 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:18.519756107 +0000 UTC m=+10.811737121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-66qtv" (UniqueName: "kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv") pod "network-check-target-v25hs" (UID: "e1b75105-2d2e-486a-b65d-2e0242c535b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:14.520206    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:14.520263    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs podName:5b615de2-b59e-417c-95d5-42cb2c7fd256 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:18.520245871 +0000 UTC m=+10.812226882 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs") pod "network-metrics-daemon-mncs6" (UID: "5b615de2-b59e-417c-95d5-42cb2c7fd256") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:14.520321    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:14.520385 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:14.520353    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret podName:2436034c-df7c-4212-b9d8-b85737f6c858 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:18.520342467 +0000 UTC m=+10.812323482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret") pod "global-pull-secret-syncer-5cxpk" (UID: "2436034c-df7c-4212-b9d8-b85737f6c858") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:14.944720 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:14.944687    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:14.945197 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:14.944833    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:15.942041 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:15.941953    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:15.942209 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:15.942086    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:15.942340 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:15.941953    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:15.942475 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:15.942439    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:16.942841 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:16.942380    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:16.942841 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:16.942529    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:17.943192 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:17.942563    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:17.943192 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:17.942712    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:17.943192 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:17.943053    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:17.943192 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:17.943155    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:18.554228 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:18.554186    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:18.554412 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:18.554251    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:18.554412 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:18.554305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:18.554412 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:18.554370    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:18.554412 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:18.554394    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:18.554412 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:18.554399    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:54:18.554667 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:18.554425    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:54:18.554667 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:18.554429    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs podName:5b615de2-b59e-417c-95d5-42cb2c7fd256 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:26.554412391 +0000 UTC m=+18.846393406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs") pod "network-metrics-daemon-mncs6" (UID: "5b615de2-b59e-417c-95d5-42cb2c7fd256") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:18.554667 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:18.554441    2575 projected.go:196] Error preparing data for projected volume kube-api-access-66qtv for pod openshift-network-diagnostics/network-check-target-v25hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:18.554667 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:18.554448    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret podName:2436034c-df7c-4212-b9d8-b85737f6c858 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:26.554438957 +0000 UTC m=+18.846419968 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret") pod "global-pull-secret-syncer-5cxpk" (UID: "2436034c-df7c-4212-b9d8-b85737f6c858") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:18.554667 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:18.554494    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv podName:e1b75105-2d2e-486a-b65d-2e0242c535b9 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:26.554477967 +0000 UTC m=+18.846458996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-66qtv" (UniqueName: "kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv") pod "network-check-target-v25hs" (UID: "e1b75105-2d2e-486a-b65d-2e0242c535b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:18.943736 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:18.943706    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:18.944218 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:18.943839    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:19.942845 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:19.942809    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:19.943033 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:19.942846    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:19.943033 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:19.942942    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:19.943151 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:19.943038    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:20.943029 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:20.942020    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:20.943029 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:20.942165    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:21.942248 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:21.942197    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:21.942421 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:21.942335    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:21.942421 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:21.942389    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:21.942538 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:21.942510    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:22.942157 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:22.942123    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:22.942578 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:22.942262    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:23.942019 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:23.941987    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:23.942227 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:23.941995    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:23.942227 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:23.942122    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:23.942227 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:23.942164    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:24.942447 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:24.942412    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:24.942845 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:24.942541    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:25.942398 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:25.942326    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:25.942398 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:25.942376    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:25.942919 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:25.942458    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:25.942919 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:25.942574    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:26.613611 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:26.613573    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:26.613798 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:26.613623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:26.613798 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:26.613674    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:26.613798 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:26.613762    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:26.613798 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:26.613771    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:54:26.613798 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:26.613796    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:54:26.613798 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:26.613797    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:26.614094 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:26.613810    2575 projected.go:196] Error preparing data for projected volume kube-api-access-66qtv for pod openshift-network-diagnostics/network-check-target-v25hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:26.614094 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:26.613829    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret podName:2436034c-df7c-4212-b9d8-b85737f6c858 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:42.613809862 +0000 UTC m=+34.905790899 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret") pod "global-pull-secret-syncer-5cxpk" (UID: "2436034c-df7c-4212-b9d8-b85737f6c858") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:26.614094 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:26.613858    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs podName:5b615de2-b59e-417c-95d5-42cb2c7fd256 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:42.613837177 +0000 UTC m=+34.905818195 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs") pod "network-metrics-daemon-mncs6" (UID: "5b615de2-b59e-417c-95d5-42cb2c7fd256") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:26.614094 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:26.613878    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv podName:e1b75105-2d2e-486a-b65d-2e0242c535b9 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:42.613867884 +0000 UTC m=+34.905848916 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-66qtv" (UniqueName: "kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv") pod "network-check-target-v25hs" (UID: "e1b75105-2d2e-486a-b65d-2e0242c535b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:26.942297 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:26.942211    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:26.942477 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:26.942362    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:27.942601 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:27.942567    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:27.943063 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:27.942679    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:27.943063 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:27.942743    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:27.943063 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:27.942849    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:28.943046 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:28.943011    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:28.943506 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:28.943106    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:29.942303 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:29.942273    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:29.942459 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:29.942284    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:29.942459 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:29.942379    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:29.942573 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:29.942489    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:30.942440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:30.942395    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:30.942976 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:30.942550    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:31.173329 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.173133    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9bgt8" event={"ID":"ffb63a5d-e35a-4530-8b14-d95314b01700","Type":"ContainerStarted","Data":"0d6130c4e2c431579374bbd692bbbae42571ac200ba9ad17fbec2f559f5f6559"}
Jul 22 12:54:31.177465 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.177433    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-khrvp" event={"ID":"91358d07-1210-40d9-a108-654aae4213f6","Type":"ContainerStarted","Data":"b036c60bbcc5c4bd378ec80ac72829c8fddf6d22d179da10401ca3749c7b254e"}
Jul 22 12:54:31.178954 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.178914    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc" event={"ID":"fac2e041-e652-495f-aaad-b007dfcc11d1","Type":"ContainerStarted","Data":"b73c8c7caf96403acf618b7bf32c8f19f82715697eba77fc3c495ee01f33664c"}
Jul 22 12:54:31.180300 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.180278    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pz4kt" event={"ID":"0ba7a0b4-cc88-4042-beb2-3c6be9e63a20","Type":"ContainerStarted","Data":"029ed0986a0bb8e8e0a1dcb8e5c7f805151baff98510fcd4cdeffdde62808c89"}
Jul 22 12:54:31.182859 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.182837    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 12:54:31.183195 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.183177    2575 generic.go:356] "Generic (PLEG): container finished" podID="0a9ea2b6-2062-4f0a-aa47-63859b3b2efd" containerID="c7b3827aae3792d6e94198c5f380aba0259004ab950cf168a732068cf4139d26" exitCode=1
Jul 22 12:54:31.183263 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.183244    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerStarted","Data":"725fcd36713b07e99d43ae8c107ed678d885a8c485a365a48a538b0c29d1c409"}
Jul 22 12:54:31.183305 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.183298    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerStarted","Data":"6a87a6ea34a572e239406b2cc1b2267bfbc040489ce5d75ed6430a726ccc4638"}
Jul 22 12:54:31.183347 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.183335    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerStarted","Data":"174d2dab12a1352b49fc06a08919c91ca0a09144120674163cc422f138e8a593"}
Jul 22 12:54:31.183389 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.183377    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerDied","Data":"c7b3827aae3792d6e94198c5f380aba0259004ab950cf168a732068cf4139d26"}
Jul 22 12:54:31.183435 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.183421    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerStarted","Data":"014285887e754f531ae7d5ed540e812a061da40d7378fce248a684da35946e08"}
Jul 22 12:54:31.184588 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.184562    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qsx49" event={"ID":"bfb3c025-7b4e-45c8-aced-0602196c44f4","Type":"ContainerStarted","Data":"40af92a6ef86e518d3f05eb802c8245603247c2d7ba37a1e7437a08346942667"}
Jul 22 12:54:31.185863 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.185845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nsbth" event={"ID":"05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055","Type":"ContainerStarted","Data":"542ad0c5cac554a0ce89c0341984374c7385a2dbd6c05289ac3c94893443d0ea"}
Jul 22 12:54:31.187411 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.187394    2575 generic.go:356] "Generic (PLEG): container finished" podID="8946923f-734c-497c-a2ee-73f244a994ee" containerID="f4d9476ea5a9384e89678d8960390dc241b6ae712afdbe069701ef8d8ba787f2" exitCode=0
Jul 22 12:54:31.187503 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.187420    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjtlj" event={"ID":"8946923f-734c-497c-a2ee-73f244a994ee","Type":"ContainerDied","Data":"f4d9476ea5a9384e89678d8960390dc241b6ae712afdbe069701ef8d8ba787f2"}
Jul 22 12:54:31.207317 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.207276    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9bgt8" podStartSLOduration=3.89223391 podStartE2EDuration="22.207261137s" totalImagesPullingTime="18.315027227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:31.193189082 +0000 UTC m=+23.485170114" watchObservedRunningTime="2026-07-22 12:54:31.207261137 +0000 UTC m=+23.499242170"
Jul 22 12:54:31.207605 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.207582    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qsx49" podStartSLOduration=3.945961021 podStartE2EDuration="22.207577643s" totalImagesPullingTime="18.261616622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:31.207116575 +0000 UTC m=+23.499097608" watchObservedRunningTime="2026-07-22 12:54:31.207577643 +0000 UTC m=+23.499558676"
Jul 22 12:54:31.240594 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.240552    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-khrvp" podStartSLOduration=3.974078264 podStartE2EDuration="22.240540737s" totalImagesPullingTime="18.266462473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:31.240486871 +0000 UTC m=+23.532467904" watchObservedRunningTime="2026-07-22 12:54:31.240540737 +0000 UTC m=+23.532521767"
Jul 22 12:54:31.253842 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.253782    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nsbth" podStartSLOduration=4.021503516 podStartE2EDuration="22.253762814s" totalImagesPullingTime="18.232259298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:31.253109578 +0000 UTC m=+23.545090613" watchObservedRunningTime="2026-07-22 12:54:31.253762814 +0000 UTC m=+23.545743850"
Jul 22 12:54:31.270035 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.269982    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pz4kt" podStartSLOduration=3.960601897 podStartE2EDuration="22.269966701s" totalImagesPullingTime="18.309364804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:31.269044022 +0000 UTC m=+23.561025076" watchObservedRunningTime="2026-07-22 12:54:31.269966701 +0000 UTC m=+23.561947734"
Jul 22 12:54:31.941919 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.941892    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:31.942082 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:31.942005    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:31.942082 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:31.942064    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:31.942175 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:31.942157    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:32.157669 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:32.157638    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 12:54:32.193156 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:32.193121    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 12:54:32.193550 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:32.193526    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerStarted","Data":"085c464f134b7b2790246956827352a46e4ce3b3313156d90265ce48cdcccbab"}
Jul 22 12:54:32.195225 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:32.195191    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d9gvw" event={"ID":"762bf3f8-33f0-4a22-93bc-18ff543c554e","Type":"ContainerStarted","Data":"06d3059fdc78a37d3b1178a47ec5ce3f1ec4b0a9e977bf8158dd33119249bc4a"}
Jul 22 12:54:32.197149 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:32.197115    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc" event={"ID":"fac2e041-e652-495f-aaad-b007dfcc11d1","Type":"ContainerStarted","Data":"5f9b0c126596108515dc75c25659dddd44d0c60e757619c017993eb538e61173"}
Jul 22 12:54:32.848321 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:32.848195    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T12:54:32.157664335Z","UUID":"d1275c3b-8e53-4429-9c28-1c67fe3a729f","Handler":null,"Name":"","Endpoint":""}
Jul 22 12:54:32.850033 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:32.850011    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 12:54:32.850033 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:32.850047    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 12:54:32.942851 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:32.942813    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:32.943049 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:32.942991    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:33.025172 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:33.025128    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:33.025861 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:33.025835    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:33.042158 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:33.042111    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d9gvw" podStartSLOduration=5.720358757 podStartE2EDuration="24.04209378s" totalImagesPullingTime="18.321735023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:32.209281441 +0000 UTC m=+24.501262476" watchObservedRunningTime="2026-07-22 12:54:33.04209378 +0000 UTC m=+25.334074816"
Jul 22 12:54:33.201496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:33.201458    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc" event={"ID":"fac2e041-e652-495f-aaad-b007dfcc11d1","Type":"ContainerStarted","Data":"cf29c94eb315575b7834043cc587d37f0a7641a48aba151d3af70cb14628e0a2"}
Jul 22 12:54:33.202205 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:33.201912    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:33.202340 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:33.202323    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-khrvp"
Jul 22 12:54:33.234562 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:33.234506    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bkwgc" podStartSLOduration=7.903739198 podStartE2EDuration="24.234484965s" totalImagesPullingTime="16.330745767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:33.219242093 +0000 UTC m=+25.511223127" watchObservedRunningTime="2026-07-22 12:54:33.234484965 +0000 UTC m=+25.526466000"
Jul 22 12:54:33.942181 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:33.942141    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:33.942181 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:33.942176    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:33.942423 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:33.942285    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:33.942423 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:33.942404    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:34.208057 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:34.207972    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 12:54:34.208480 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:34.208410    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerStarted","Data":"36c2067744295f1fbdd765effe3f9843ced1934f622a468e83e9a08ac8aff0e8"}
Jul 22 12:54:34.942844 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:34.942599    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:34.943045 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:34.942981    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:35.942670 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:35.942492    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:35.943391 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:35.942492    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:35.943391 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:35.942758    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:35.943391 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:35.942829    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:36.198737 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.198710    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qsx49_bfb3c025-7b4e-45c8-aced-0602196c44f4/dns-node-resolver/0.log"
Jul 22 12:54:36.215235 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.215207    2575 generic.go:356] "Generic (PLEG): container finished" podID="8946923f-734c-497c-a2ee-73f244a994ee" containerID="dfbdcfc2854661217d77bfb11e36c8205a1329343ea86b27d17149c9a68d7e02" exitCode=0
Jul 22 12:54:36.215366 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.215299    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjtlj" event={"ID":"8946923f-734c-497c-a2ee-73f244a994ee","Type":"ContainerDied","Data":"dfbdcfc2854661217d77bfb11e36c8205a1329343ea86b27d17149c9a68d7e02"}
Jul 22 12:54:36.218767 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.218751    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 12:54:36.219168 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.219150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerStarted","Data":"de4b4df878798514c493f75663738dab8123dadbb3d4970bacc4430f046f8673"}
Jul 22 12:54:36.219431 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.219401    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:36.219503 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.219465    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:36.219592 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.219576    2575 scope.go:117] "RemoveContainer" containerID="c7b3827aae3792d6e94198c5f380aba0259004ab950cf168a732068cf4139d26"
Jul 22 12:54:36.236636 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.236616    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:36.779084 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.779059    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nsbth_05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055/node-ca/0.log"
Jul 22 12:54:36.941856 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:36.941831    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:36.942012 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:36.941942    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:37.223831 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.223799    2575 generic.go:356] "Generic (PLEG): container finished" podID="8946923f-734c-497c-a2ee-73f244a994ee" containerID="5b6b2bc47d0f6689242bc9363d15fa16f7f0961053f919fe796b1f49a1603888" exitCode=0
Jul 22 12:54:37.224267 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.223880    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjtlj" event={"ID":"8946923f-734c-497c-a2ee-73f244a994ee","Type":"ContainerDied","Data":"5b6b2bc47d0f6689242bc9363d15fa16f7f0961053f919fe796b1f49a1603888"}
Jul 22 12:54:37.227991 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.227973    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 12:54:37.228368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.228345    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" event={"ID":"0a9ea2b6-2062-4f0a-aa47-63859b3b2efd","Type":"ContainerStarted","Data":"ec21544ecf46e922fb6f479772be7d183203183a192a7ba2f113b5475f32fbcc"}
Jul 22 12:54:37.228733 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.228715    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:37.245425 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.245388    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:54:37.274825 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.274775    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v" podStartSLOduration=9.893882102 podStartE2EDuration="28.274759169s" totalImagesPullingTime="18.380877067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:37.273264679 +0000 UTC m=+29.565245711" watchObservedRunningTime="2026-07-22 12:54:37.274759169 +0000 UTC m=+29.566740228"
Jul 22 12:54:37.574462 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.574421    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v25hs"]
Jul 22 12:54:37.574623 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.574549    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:37.574679 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:37.574652    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:37.578003 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.577974    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mncs6"]
Jul 22 12:54:37.578155 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.578087    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:37.578220 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:37.578203    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:37.578554 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.578535    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5cxpk"]
Jul 22 12:54:37.578656 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:37.578635    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:37.578743 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:37.578724    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:38.232915 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:38.232879    2575 generic.go:356] "Generic (PLEG): container finished" podID="8946923f-734c-497c-a2ee-73f244a994ee" containerID="71e101e767f4da67621392976ed236d4e9300fcc45db590c611c2c7b1795d0f8" exitCode=0
Jul 22 12:54:38.233366 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:38.232962    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjtlj" event={"ID":"8946923f-734c-497c-a2ee-73f244a994ee","Type":"ContainerDied","Data":"71e101e767f4da67621392976ed236d4e9300fcc45db590c611c2c7b1795d0f8"}
Jul 22 12:54:38.943597 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:38.943400    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:38.943824 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:38.943532    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:38.943824 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:38.943650    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:38.943824 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:38.943732    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:38.944002 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:38.943894    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:38.944052 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:38.944034    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:40.942225 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:40.942180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:40.942716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:40.942180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:40.942716 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:40.942313    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:40.942716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:40.942177    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:40.942716 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:40.942396    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:40.942716 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:40.942460    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:42.636338 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:42.636294    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:42.636352    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:42.636394    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.636452    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.636506    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.636532    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.636550    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.636563    2575 projected.go:196] Error preparing data for projected volume kube-api-access-66qtv for pod openshift-network-diagnostics/network-check-target-v25hs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.636536    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret podName:2436034c-df7c-4212-b9d8-b85737f6c858 nodeName:}" failed. No retries permitted until 2026-07-22 12:55:14.636516584 +0000 UTC m=+66.928497596 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret") pod "global-pull-secret-syncer-5cxpk" (UID: "2436034c-df7c-4212-b9d8-b85737f6c858") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.636612    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs podName:5b615de2-b59e-417c-95d5-42cb2c7fd256 nodeName:}" failed. No retries permitted until 2026-07-22 12:55:14.636596879 +0000 UTC m=+66.928577905 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs") pod "network-metrics-daemon-mncs6" (UID: "5b615de2-b59e-417c-95d5-42cb2c7fd256") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:54:42.636892 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.636628    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv podName:e1b75105-2d2e-486a-b65d-2e0242c535b9 nodeName:}" failed. No retries permitted until 2026-07-22 12:55:14.636618684 +0000 UTC m=+66.928599701 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-66qtv" (UniqueName: "kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv") pod "network-check-target-v25hs" (UID: "e1b75105-2d2e-486a-b65d-2e0242c535b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:54:42.941919 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:42.941846    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:42.941919 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:42.941862    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:42.941919 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:42.941871    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:42.942236 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.941990    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5cxpk" podUID="2436034c-df7c-4212-b9d8-b85737f6c858"
Jul 22 12:54:42.942236 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.942136    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mncs6" podUID="5b615de2-b59e-417c-95d5-42cb2c7fd256"
Jul 22 12:54:42.942236 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:42.942225    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v25hs" podUID="e1b75105-2d2e-486a-b65d-2e0242c535b9"
Jul 22 12:54:43.440577 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.440549    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-133.ec2.internal" event="NodeReady"
Jul 22 12:54:43.440836 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.440678    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 12:54:43.484484 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.484450    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9p9bf"]
Jul 22 12:54:43.510214 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.510186    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v6nbv"]
Jul 22 12:54:43.510366 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.510294    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.512922 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.512900    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6s9tz\""
Jul 22 12:54:43.513056 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.512946    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 12:54:43.513170 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.513157    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 12:54:43.535465 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.535442    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9p9bf"]
Jul 22 12:54:43.535575 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.535474    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v6nbv"]
Jul 22 12:54:43.535575 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.535488    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dh668"]
Jul 22 12:54:43.535575 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.535562    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v6nbv"
Jul 22 12:54:43.537782 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.537758    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 12:54:43.537911 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.537850    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 12:54:43.538169 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.538152    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-x9ndv\""
Jul 22 12:54:43.538275 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.538197    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 12:54:43.557894 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.557872    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dh668"]
Jul 22 12:54:43.558009 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.557997    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.560520 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.560501    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dxxk2\""
Jul 22 12:54:43.560520 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.560514    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 12:54:43.560653 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.560644    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 12:54:43.560756 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.560742    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 12:54:43.560819 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.560772    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 12:54:43.642239 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642212    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lphbw\" (UniqueName: \"kubernetes.io/projected/e5d9191b-4abb-4410-a96c-a2665c513b75-kube-api-access-lphbw\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ae71a63-bc40-4528-8ce5-626c8d3daf02-crio-socket\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642290    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zk27\" (UniqueName: \"kubernetes.io/projected/6ae71a63-bc40-4528-8ce5-626c8d3daf02-kube-api-access-9zk27\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642366    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wg8s\" (UniqueName: \"kubernetes.io/projected/1dd6eef4-46dd-469e-b5e0-49f2b5cc581b-kube-api-access-7wg8s\") pod \"ingress-canary-v6nbv\" (UID: \"1dd6eef4-46dd-469e-b5e0-49f2b5cc581b\") " pod="openshift-ingress-canary/ingress-canary-v6nbv"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642396    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642419    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5d9191b-4abb-4410-a96c-a2665c513b75-tmp-dir\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1dd6eef4-46dd-469e-b5e0-49f2b5cc581b-cert\") pod \"ingress-canary-v6nbv\" (UID: \"1dd6eef4-46dd-469e-b5e0-49f2b5cc581b\") " pod="openshift-ingress-canary/ingress-canary-v6nbv"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642480    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ae71a63-bc40-4528-8ce5-626c8d3daf02-data-volume\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642552    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ae71a63-bc40-4528-8ce5-626c8d3daf02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642600    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5d9191b-4abb-4410-a96c-a2665c513b75-config-volume\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.642766 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.642628    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.743528 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743503    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1dd6eef4-46dd-469e-b5e0-49f2b5cc581b-cert\") pod \"ingress-canary-v6nbv\" (UID: \"1dd6eef4-46dd-469e-b5e0-49f2b5cc581b\") " pod="openshift-ingress-canary/ingress-canary-v6nbv"
Jul 22 12:54:43.743650 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ae71a63-bc40-4528-8ce5-626c8d3daf02-data-volume\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.743650 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743551    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ae71a63-bc40-4528-8ce5-626c8d3daf02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.743650 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743573    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5d9191b-4abb-4410-a96c-a2665c513b75-config-volume\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.743808 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.743808 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lphbw\" (UniqueName: \"kubernetes.io/projected/e5d9191b-4abb-4410-a96c-a2665c513b75-kube-api-access-lphbw\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.743808 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743797    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ae71a63-bc40-4528-8ce5-626c8d3daf02-crio-socket\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.743954 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743826    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zk27\" (UniqueName: \"kubernetes.io/projected/6ae71a63-bc40-4528-8ce5-626c8d3daf02-kube-api-access-9zk27\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.743954 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:43.743833    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:54:43.743954 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743873    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wg8s\" (UniqueName: \"kubernetes.io/projected/1dd6eef4-46dd-469e-b5e0-49f2b5cc581b-kube-api-access-7wg8s\") pod \"ingress-canary-v6nbv\" (UID: \"1dd6eef4-46dd-469e-b5e0-49f2b5cc581b\") " pod="openshift-ingress-canary/ingress-canary-v6nbv"
Jul 22 12:54:43.743954 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.743889    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ae71a63-bc40-4528-8ce5-626c8d3daf02-data-volume\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.743954 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:43.743913    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls podName:e5d9191b-4abb-4410-a96c-a2665c513b75 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:44.243882609 +0000 UTC m=+36.535863620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls") pod "dns-default-9p9bf" (UID: "e5d9191b-4abb-4410-a96c-a2665c513b75") : secret "dns-default-metrics-tls" not found
Jul 22 12:54:43.744167 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.744095    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ae71a63-bc40-4528-8ce5-626c8d3daf02-crio-socket\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.744221 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.744212    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5d9191b-4abb-4410-a96c-a2665c513b75-config-volume\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.744272 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.744242    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ae71a63-bc40-4528-8ce5-626c8d3daf02-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.744375 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.744355    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.744417 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.744386    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5d9191b-4abb-4410-a96c-a2665c513b75-tmp-dir\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.744486 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:43.744467    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:43.744600 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:43.744587    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls podName:6ae71a63-bc40-4528-8ce5-626c8d3daf02 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:44.244565404 +0000 UTC m=+36.536546427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dh668" (UID: "6ae71a63-bc40-4528-8ce5-626c8d3daf02") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:43.744646 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.744633    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5d9191b-4abb-4410-a96c-a2665c513b75-tmp-dir\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.747971 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.747949    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1dd6eef4-46dd-469e-b5e0-49f2b5cc581b-cert\") pod \"ingress-canary-v6nbv\" (UID: \"1dd6eef4-46dd-469e-b5e0-49f2b5cc581b\") " pod="openshift-ingress-canary/ingress-canary-v6nbv"
Jul 22 12:54:43.753860 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.753836    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wg8s\" (UniqueName: \"kubernetes.io/projected/1dd6eef4-46dd-469e-b5e0-49f2b5cc581b-kube-api-access-7wg8s\") pod \"ingress-canary-v6nbv\" (UID: \"1dd6eef4-46dd-469e-b5e0-49f2b5cc581b\") " pod="openshift-ingress-canary/ingress-canary-v6nbv"
Jul 22 12:54:43.753990 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.753873    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zk27\" (UniqueName: \"kubernetes.io/projected/6ae71a63-bc40-4528-8ce5-626c8d3daf02-kube-api-access-9zk27\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:43.754086 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.754061    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lphbw\" (UniqueName: \"kubernetes.io/projected/e5d9191b-4abb-4410-a96c-a2665c513b75-kube-api-access-lphbw\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:43.843386 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:43.843357    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v6nbv"
Jul 22 12:54:44.046147 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.045915    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v6nbv"]
Jul 22 12:54:44.048637 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:44.048608    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd6eef4_46dd_469e_b5e0_49f2b5cc581b.slice/crio-b98ad918fccdafab7855b58b45d5e014f4cde3b6f156a69b0b9108d5150bd053 WatchSource:0}: Error finding container b98ad918fccdafab7855b58b45d5e014f4cde3b6f156a69b0b9108d5150bd053: Status 404 returned error can't find the container with id b98ad918fccdafab7855b58b45d5e014f4cde3b6f156a69b0b9108d5150bd053
Jul 22 12:54:44.248853 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.248775    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:44.249024 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.248854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:44.249024 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:44.248905    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:54:44.249024 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:44.248977    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:44.249024 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:44.248983    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls podName:e5d9191b-4abb-4410-a96c-a2665c513b75 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:45.248967539 +0000 UTC m=+37.540948557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls") pod "dns-default-9p9bf" (UID: "e5d9191b-4abb-4410-a96c-a2665c513b75") : secret "dns-default-metrics-tls" not found
Jul 22 12:54:44.249183 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:44.249034    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls podName:6ae71a63-bc40-4528-8ce5-626c8d3daf02 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:45.249020895 +0000 UTC m=+37.541001906 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dh668" (UID: "6ae71a63-bc40-4528-8ce5-626c8d3daf02") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:44.249478 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.249456    2575 generic.go:356] "Generic (PLEG): container finished" podID="8946923f-734c-497c-a2ee-73f244a994ee" containerID="4460a81192b09bacf0f0fb502471f0fa3caa9d6ed43169c32835204eb0971da4" exitCode=0
Jul 22 12:54:44.249553 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.249532    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjtlj" event={"ID":"8946923f-734c-497c-a2ee-73f244a994ee","Type":"ContainerDied","Data":"4460a81192b09bacf0f0fb502471f0fa3caa9d6ed43169c32835204eb0971da4"}
Jul 22 12:54:44.250745 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.250571    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v6nbv" event={"ID":"1dd6eef4-46dd-469e-b5e0-49f2b5cc581b","Type":"ContainerStarted","Data":"b98ad918fccdafab7855b58b45d5e014f4cde3b6f156a69b0b9108d5150bd053"}
Jul 22 12:54:44.942231 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.942188    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:54:44.942899 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.942194    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:54:44.942899 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.942194    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:54:44.945164 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.945115    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 12:54:44.946254 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.946236    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 12:54:44.946363 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.946311    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fsqrc\""
Jul 22 12:54:44.946363 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.946350    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 12:54:44.946453 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.946356    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 12:54:44.946508 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:44.946481    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-frnxs\""
Jul 22 12:54:45.255429 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.255355    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:45.255612 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.255445    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:45.255612 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:45.255509    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:54:45.255612 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:45.255552    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:45.255612 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:45.255574    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls podName:e5d9191b-4abb-4410-a96c-a2665c513b75 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:47.255552166 +0000 UTC m=+39.547533197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls") pod "dns-default-9p9bf" (UID: "e5d9191b-4abb-4410-a96c-a2665c513b75") : secret "dns-default-metrics-tls" not found
Jul 22 12:54:45.255612 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:45.255602    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls podName:6ae71a63-bc40-4528-8ce5-626c8d3daf02 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:47.255585178 +0000 UTC m=+39.547566225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dh668" (UID: "6ae71a63-bc40-4528-8ce5-626c8d3daf02") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:45.257112 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.257087    2575 generic.go:356] "Generic (PLEG): container finished" podID="8946923f-734c-497c-a2ee-73f244a994ee" containerID="6ac6b395a1d9f08d47a22dac4f839d32df6209711e1dcd1ee7500a8934bdc2f7" exitCode=0
Jul 22 12:54:45.257248 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.257141    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjtlj" event={"ID":"8946923f-734c-497c-a2ee-73f244a994ee","Type":"ContainerDied","Data":"6ac6b395a1d9f08d47a22dac4f839d32df6209711e1dcd1ee7500a8934bdc2f7"}
Jul 22 12:54:45.350255 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.350230    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9"]
Jul 22 12:54:45.383250 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.383213    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9"]
Jul 22 12:54:45.383400 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.383326    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9"
Jul 22 12:54:45.386389 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.386197    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 22 12:54:45.386389 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.386276    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wwq8q\""
Jul 22 12:54:45.386389 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.386311    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 22 12:54:45.456595 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.456561    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq7w5\" (UniqueName: \"kubernetes.io/projected/3b2461d0-33da-4111-a51f-be731413653a-kube-api-access-sq7w5\") pod \"migrator-7cc9c7ffd7-2vtp9\" (UID: \"3b2461d0-33da-4111-a51f-be731413653a\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9"
Jul 22 12:54:45.557838 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.557753    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq7w5\" (UniqueName: \"kubernetes.io/projected/3b2461d0-33da-4111-a51f-be731413653a-kube-api-access-sq7w5\") pod \"migrator-7cc9c7ffd7-2vtp9\" (UID: \"3b2461d0-33da-4111-a51f-be731413653a\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9"
Jul 22 12:54:45.569041 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.569010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq7w5\" (UniqueName: \"kubernetes.io/projected/3b2461d0-33da-4111-a51f-be731413653a-kube-api-access-sq7w5\") pod \"migrator-7cc9c7ffd7-2vtp9\" (UID: \"3b2461d0-33da-4111-a51f-be731413653a\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9"
Jul 22 12:54:45.693435 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.693401    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9"
Jul 22 12:54:45.841497 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:45.841283    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9"]
Jul 22 12:54:45.845797 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:45.845769    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2461d0_33da_4111_a51f_be731413653a.slice/crio-d809adc96acee8110a79436fd217796852e195ca611bbacf8457ea0fbb982478 WatchSource:0}: Error finding container d809adc96acee8110a79436fd217796852e195ca611bbacf8457ea0fbb982478: Status 404 returned error can't find the container with id d809adc96acee8110a79436fd217796852e195ca611bbacf8457ea0fbb982478
Jul 22 12:54:46.261604 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:46.261523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9" event={"ID":"3b2461d0-33da-4111-a51f-be731413653a","Type":"ContainerStarted","Data":"d809adc96acee8110a79436fd217796852e195ca611bbacf8457ea0fbb982478"}
Jul 22 12:54:46.265709 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:46.265679    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjtlj" event={"ID":"8946923f-734c-497c-a2ee-73f244a994ee","Type":"ContainerStarted","Data":"7f348b1da688491773e8bfeeed5d61b67d3eea5ab391c0ea30b13ca470b79fe4"}
Jul 22 12:54:46.293746 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:46.293691    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wjtlj" podStartSLOduration=7.65945986 podStartE2EDuration="37.29366991s" totalImagesPullingTime="29.63421005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:46.29099435 +0000 UTC m=+38.582975386" watchObservedRunningTime="2026-07-22 12:54:46.29366991 +0000 UTC m=+38.585650944"
Jul 22 12:54:47.269291 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.269201    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:47.270024 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:47.269359    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:54:47.270024 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.269416    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:47.270024 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:47.269422    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls podName:e5d9191b-4abb-4410-a96c-a2665c513b75 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:51.269403217 +0000 UTC m=+43.561384231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls") pod "dns-default-9p9bf" (UID: "e5d9191b-4abb-4410-a96c-a2665c513b75") : secret "dns-default-metrics-tls" not found
Jul 22 12:54:47.270024 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:47.269539    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:47.270024 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:47.269594    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls podName:6ae71a63-bc40-4528-8ce5-626c8d3daf02 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:51.269577136 +0000 UTC m=+43.561558148 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dh668" (UID: "6ae71a63-bc40-4528-8ce5-626c8d3daf02") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:47.270211 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.270134    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v6nbv" event={"ID":"1dd6eef4-46dd-469e-b5e0-49f2b5cc581b","Type":"ContainerStarted","Data":"97960264f460535f0ecc8533ce3442d9633e755f3376d04668bd046a8ff853f0"}
Jul 22 12:54:47.286322 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.286278    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v6nbv" podStartSLOduration=1.461758972 podStartE2EDuration="4.286263521s" totalImagesPullingTime="2.824504549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:47.285624826 +0000 UTC m=+39.577605858" watchObservedRunningTime="2026-07-22 12:54:47.286263521 +0000 UTC m=+39.578244553"
Jul 22 12:54:47.561676 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.561603    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-w4jkd"]
Jul 22 12:54:47.569889 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.569861    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.572383 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.572354    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 22 12:54:47.572510 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.572355    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 22 12:54:47.572510 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.572355    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 22 12:54:47.572510 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.572463    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wgrlg\""
Jul 22 12:54:47.572510 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.572495    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 22 12:54:47.574026 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.574004    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-w4jkd"]
Jul 22 12:54:47.672815 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.672784    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7hwm\" (UniqueName: \"kubernetes.io/projected/81f053ee-61e5-4730-a1fc-73502f926c9d-kube-api-access-l7hwm\") pod \"service-ca-58598695f7-w4jkd\" (UID: \"81f053ee-61e5-4730-a1fc-73502f926c9d\") " pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.672984 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.672840    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/81f053ee-61e5-4730-a1fc-73502f926c9d-signing-key\") pod \"service-ca-58598695f7-w4jkd\" (UID: \"81f053ee-61e5-4730-a1fc-73502f926c9d\") " pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.672984 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.672943    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/81f053ee-61e5-4730-a1fc-73502f926c9d-signing-cabundle\") pod \"service-ca-58598695f7-w4jkd\" (UID: \"81f053ee-61e5-4730-a1fc-73502f926c9d\") " pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.773697 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.773661    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/81f053ee-61e5-4730-a1fc-73502f926c9d-signing-key\") pod \"service-ca-58598695f7-w4jkd\" (UID: \"81f053ee-61e5-4730-a1fc-73502f926c9d\") " pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.773871 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.773714    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/81f053ee-61e5-4730-a1fc-73502f926c9d-signing-cabundle\") pod \"service-ca-58598695f7-w4jkd\" (UID: \"81f053ee-61e5-4730-a1fc-73502f926c9d\") " pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.773971 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.773885    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7hwm\" (UniqueName: \"kubernetes.io/projected/81f053ee-61e5-4730-a1fc-73502f926c9d-kube-api-access-l7hwm\") pod \"service-ca-58598695f7-w4jkd\" (UID: \"81f053ee-61e5-4730-a1fc-73502f926c9d\") " pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.774328 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.774311    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/81f053ee-61e5-4730-a1fc-73502f926c9d-signing-cabundle\") pod \"service-ca-58598695f7-w4jkd\" (UID: \"81f053ee-61e5-4730-a1fc-73502f926c9d\") " pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.776997 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.776970    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/81f053ee-61e5-4730-a1fc-73502f926c9d-signing-key\") pod \"service-ca-58598695f7-w4jkd\" (UID: \"81f053ee-61e5-4730-a1fc-73502f926c9d\") " pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.783771 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.783748    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7hwm\" (UniqueName: \"kubernetes.io/projected/81f053ee-61e5-4730-a1fc-73502f926c9d-kube-api-access-l7hwm\") pod \"service-ca-58598695f7-w4jkd\" (UID: \"81f053ee-61e5-4730-a1fc-73502f926c9d\") " pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:47.880636 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:47.880613    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-w4jkd"
Jul 22 12:54:48.050716 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:48.035360    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-w4jkd"]
Jul 22 12:54:48.070850 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:48.070823    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f053ee_61e5_4730_a1fc_73502f926c9d.slice/crio-3c42ac9858292d2ba56ef95effd6af9c84bedac29e9db8bff331a58a7fabfbbf WatchSource:0}: Error finding container 3c42ac9858292d2ba56ef95effd6af9c84bedac29e9db8bff331a58a7fabfbbf: Status 404 returned error can't find the container with id 3c42ac9858292d2ba56ef95effd6af9c84bedac29e9db8bff331a58a7fabfbbf
Jul 22 12:54:48.273727 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:48.273692    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9" event={"ID":"3b2461d0-33da-4111-a51f-be731413653a","Type":"ContainerStarted","Data":"2887bec002ccd5c704592698efdbcc03d2dd49e8cc7a5fd7eb7396dafa38f29d"}
Jul 22 12:54:48.274237 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:48.273754    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9" event={"ID":"3b2461d0-33da-4111-a51f-be731413653a","Type":"ContainerStarted","Data":"f3f6f4cfe85a3937180e12779032b3cb85ed6512c2270c2404af77b9dfde1d6e"}
Jul 22 12:54:48.274957 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:48.274920    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-w4jkd" event={"ID":"81f053ee-61e5-4730-a1fc-73502f926c9d","Type":"ContainerStarted","Data":"3c42ac9858292d2ba56ef95effd6af9c84bedac29e9db8bff331a58a7fabfbbf"}
Jul 22 12:54:48.289484 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:48.289443    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-2vtp9" podStartSLOduration=1.287293284 podStartE2EDuration="3.289429853s" totalImagesPullingTime="2.002136569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:54:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:48.289394975 +0000 UTC m=+40.581376041" watchObservedRunningTime="2026-07-22 12:54:48.289429853 +0000 UTC m=+40.581410887"
Jul 22 12:54:51.284059 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:51.284022    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-w4jkd" event={"ID":"81f053ee-61e5-4730-a1fc-73502f926c9d","Type":"ContainerStarted","Data":"e46edc3c108df6e5f65c9b9d970826975fd1ba0c57ea5e337a001034eb6dda27"}
Jul 22 12:54:51.304219 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:51.304185    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:51.305087 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:51.305036    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-w4jkd" podStartSLOduration=2.031129908 podStartE2EDuration="4.305015674s" totalImagesPullingTime="2.273885766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:54:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:54:51.304305362 +0000 UTC m=+43.596286428" watchObservedRunningTime="2026-07-22 12:54:51.305015674 +0000 UTC m=+43.596996707"
Jul 22 12:54:51.305295 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:51.305270    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:51.305363 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:51.305298    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:51.305415 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:51.305371    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls podName:6ae71a63-bc40-4528-8ce5-626c8d3daf02 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:59.305351288 +0000 UTC m=+51.597332322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dh668" (UID: "6ae71a63-bc40-4528-8ce5-626c8d3daf02") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:54:51.305470 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:51.305432    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:54:51.305535 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:54:51.305475    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls podName:e5d9191b-4abb-4410-a96c-a2665c513b75 nodeName:}" failed. No retries permitted until 2026-07-22 12:54:59.305462565 +0000 UTC m=+51.597443590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls") pod "dns-default-9p9bf" (UID: "e5d9191b-4abb-4410-a96c-a2665c513b75") : secret "dns-default-metrics-tls" not found
Jul 22 12:54:59.368799 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:59.368755    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:59.369322 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:59.368834    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:59.372212 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:59.372182    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5d9191b-4abb-4410-a96c-a2665c513b75-metrics-tls\") pod \"dns-default-9p9bf\" (UID: \"e5d9191b-4abb-4410-a96c-a2665c513b75\") " pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:59.372341 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:59.372216    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ae71a63-bc40-4528-8ce5-626c8d3daf02-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dh668\" (UID: \"6ae71a63-bc40-4528-8ce5-626c8d3daf02\") " pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:59.420154 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:59.420114    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:54:59.465839 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:59.465810    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dh668"
Jul 22 12:54:59.562543 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:59.562459    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9p9bf"]
Jul 22 12:54:59.565670 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:59.565634    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d9191b_4abb_4410_a96c_a2665c513b75.slice/crio-62f55561e84007e08efd342836a8485ccd97555b2020a86d1e3f941a6f6b0ba9 WatchSource:0}: Error finding container 62f55561e84007e08efd342836a8485ccd97555b2020a86d1e3f941a6f6b0ba9: Status 404 returned error can't find the container with id 62f55561e84007e08efd342836a8485ccd97555b2020a86d1e3f941a6f6b0ba9
Jul 22 12:54:59.616026 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:54:59.615991    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dh668"]
Jul 22 12:54:59.619059 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:54:59.619003    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae71a63_bc40_4528_8ce5_626c8d3daf02.slice/crio-52c0503ef3dc7623fbc0b25c30a957a3e43d34745d59970c7f57df0b848db84d WatchSource:0}: Error finding container 52c0503ef3dc7623fbc0b25c30a957a3e43d34745d59970c7f57df0b848db84d: Status 404 returned error can't find the container with id 52c0503ef3dc7623fbc0b25c30a957a3e43d34745d59970c7f57df0b848db84d
Jul 22 12:55:00.309555 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:00.309473    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dh668" event={"ID":"6ae71a63-bc40-4528-8ce5-626c8d3daf02","Type":"ContainerStarted","Data":"52c0503ef3dc7623fbc0b25c30a957a3e43d34745d59970c7f57df0b848db84d"}
Jul 22 12:55:00.314619 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:00.312849    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9p9bf" event={"ID":"e5d9191b-4abb-4410-a96c-a2665c513b75","Type":"ContainerStarted","Data":"62f55561e84007e08efd342836a8485ccd97555b2020a86d1e3f941a6f6b0ba9"}
Jul 22 12:55:02.323679 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:02.323652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9p9bf" event={"ID":"e5d9191b-4abb-4410-a96c-a2665c513b75","Type":"ContainerStarted","Data":"e08a354c7719a45c0154028d231b1fcbcf6f14d2cc11712a420e677f8f920be5"}
Jul 22 12:55:05.335083 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:05.335045    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dh668" event={"ID":"6ae71a63-bc40-4528-8ce5-626c8d3daf02","Type":"ContainerStarted","Data":"7ed3f6ea8193f9393dfd9cb41fbf010b8ac7172b8cf6e6482b4a57839ddfdf92"}
Jul 22 12:55:05.336818 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:05.336787    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9p9bf" event={"ID":"e5d9191b-4abb-4410-a96c-a2665c513b75","Type":"ContainerStarted","Data":"0f5fb30aa5e44a421d7a31efce7c6ea68af1e55423156c3e72cd4fab50841744"}
Jul 22 12:55:05.336947 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:05.336860    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:55:05.354392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:05.354344    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9p9bf" podStartSLOduration=19.920097439 podStartE2EDuration="22.354329629s" totalImagesPullingTime="2.43423219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:05.353702018 +0000 UTC m=+57.645683052" watchObservedRunningTime="2026-07-22 12:55:05.354329629 +0000 UTC m=+57.646310661"
Jul 22 12:55:09.247155 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.247127    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8g9v"
Jul 22 12:55:09.351759 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.351720    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dh668" event={"ID":"6ae71a63-bc40-4528-8ce5-626c8d3daf02","Type":"ContainerStarted","Data":"b8b1e1ab6950d3be1253bc2ee71d0a39aafea45655285ad235e71b3ae1489cb5"}
Jul 22 12:55:09.444322 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.444284    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"]
Jul 22 12:55:09.447388 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.447356    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"
Jul 22 12:55:09.452424 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.452396    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 22 12:55:09.452601 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.452396    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4tf6m\""
Jul 22 12:55:09.452709 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.452629    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 22 12:55:09.452990 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.452857    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 22 12:55:09.452990 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.452975    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 22 12:55:09.458753 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.458716    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"]
Jul 22 12:55:09.542633 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.542550    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-rt7l9"]
Jul 22 12:55:09.545437 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.545415    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-rt7l9"
Jul 22 12:55:09.548135 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.548113    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-6j2ql\""
Jul 22 12:55:09.548314 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.548113    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 22 12:55:09.548314 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.548204    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 22 12:55:09.550602 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.550495    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34497cf8-da5e-4c4d-b977-32821c24d09e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt\" (UID: \"34497cf8-da5e-4c4d-b977-32821c24d09e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"
Jul 22 12:55:09.550602 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.550531    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwwhq\" (UniqueName: \"kubernetes.io/projected/34497cf8-da5e-4c4d-b977-32821c24d09e-kube-api-access-vwwhq\") pod \"managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt\" (UID: \"34497cf8-da5e-4c4d-b977-32821c24d09e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"
Jul 22 12:55:09.552178 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.552157    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d49b55997-brbvr"]
Jul 22 12:55:09.555187 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.555167    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"]
Jul 22 12:55:09.555450 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.555431    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.557916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.557897    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9xmtd\""
Jul 22 12:55:09.557916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.557909    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 12:55:09.558082 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.557909    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 12:55:09.558232 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.558210    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-rt7l9"]
Jul 22 12:55:09.558232 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.558256    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 12:55:09.559496 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.558301    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"
Jul 22 12:55:09.561565 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.561490    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 22 12:55:09.561795 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.561779    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4npdd\""
Jul 22 12:55:09.564986 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.564295    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 12:55:09.569263 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.569173    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"]
Jul 22 12:55:09.572253 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.572231    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d49b55997-brbvr"]
Jul 22 12:55:09.651546 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51c80405-cd6f-43de-89ad-120b15b6b0ce-registry-certificates\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.651714 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651564    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51c80405-cd6f-43de-89ad-120b15b6b0ce-ca-trust-extracted\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.651714 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651591    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-686nq\" (UniqueName: \"kubernetes.io/projected/51c80405-cd6f-43de-89ad-120b15b6b0ce-kube-api-access-686nq\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.651714 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651619    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51c80405-cd6f-43de-89ad-120b15b6b0ce-image-registry-private-configuration\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.651714 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0502af9d-e8c9-4ea7-92a1-c2d11aa10d48-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wfrwl\" (UID: \"0502af9d-e8c9-4ea7-92a1-c2d11aa10d48\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"
Jul 22 12:55:09.651882 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51c80405-cd6f-43de-89ad-120b15b6b0ce-registry-tls\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.651882 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgq7l\" (UniqueName: \"kubernetes.io/projected/cc75880f-e47d-40b0-aded-055b41bc1a02-kube-api-access-xgq7l\") pod \"downloads-5bc67dcd46-rt7l9\" (UID: \"cc75880f-e47d-40b0-aded-055b41bc1a02\") " pod="openshift-console/downloads-5bc67dcd46-rt7l9"
Jul 22 12:55:09.651882 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51c80405-cd6f-43de-89ad-120b15b6b0ce-trusted-ca\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.651882 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651842    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34497cf8-da5e-4c4d-b977-32821c24d09e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt\" (UID: \"34497cf8-da5e-4c4d-b977-32821c24d09e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"
Jul 22 12:55:09.651882 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651859    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51c80405-cd6f-43de-89ad-120b15b6b0ce-installation-pull-secrets\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.651882 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651873    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51c80405-cd6f-43de-89ad-120b15b6b0ce-bound-sa-token\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.652119 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.651895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwwhq\" (UniqueName: \"kubernetes.io/projected/34497cf8-da5e-4c4d-b977-32821c24d09e-kube-api-access-vwwhq\") pod \"managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt\" (UID: \"34497cf8-da5e-4c4d-b977-32821c24d09e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"
Jul 22 12:55:09.654876 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.654846    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34497cf8-da5e-4c4d-b977-32821c24d09e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt\" (UID: \"34497cf8-da5e-4c4d-b977-32821c24d09e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"
Jul 22 12:55:09.667547 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.667519    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwwhq\" (UniqueName: \"kubernetes.io/projected/34497cf8-da5e-4c4d-b977-32821c24d09e-kube-api-access-vwwhq\") pod \"managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt\" (UID: \"34497cf8-da5e-4c4d-b977-32821c24d09e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"
Jul 22 12:55:09.752707 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752674    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51c80405-cd6f-43de-89ad-120b15b6b0ce-registry-certificates\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.752707 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51c80405-cd6f-43de-89ad-120b15b6b0ce-ca-trust-extracted\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.752969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752745    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-686nq\" (UniqueName: \"kubernetes.io/projected/51c80405-cd6f-43de-89ad-120b15b6b0ce-kube-api-access-686nq\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.752969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752775    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51c80405-cd6f-43de-89ad-120b15b6b0ce-image-registry-private-configuration\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.752969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752814    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0502af9d-e8c9-4ea7-92a1-c2d11aa10d48-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wfrwl\" (UID: \"0502af9d-e8c9-4ea7-92a1-c2d11aa10d48\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"
Jul 22 12:55:09.752969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752842    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51c80405-cd6f-43de-89ad-120b15b6b0ce-registry-tls\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.752969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgq7l\" (UniqueName: \"kubernetes.io/projected/cc75880f-e47d-40b0-aded-055b41bc1a02-kube-api-access-xgq7l\") pod \"downloads-5bc67dcd46-rt7l9\" (UID: \"cc75880f-e47d-40b0-aded-055b41bc1a02\") " pod="openshift-console/downloads-5bc67dcd46-rt7l9"
Jul 22 12:55:09.752969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752918    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51c80405-cd6f-43de-89ad-120b15b6b0ce-trusted-ca\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.752969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752956    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51c80405-cd6f-43de-89ad-120b15b6b0ce-installation-pull-secrets\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.752969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.752973    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51c80405-cd6f-43de-89ad-120b15b6b0ce-bound-sa-token\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.753863 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.753830    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51c80405-cd6f-43de-89ad-120b15b6b0ce-registry-certificates\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.754196 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.754173    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51c80405-cd6f-43de-89ad-120b15b6b0ce-trusted-ca\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.754337 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.754309    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51c80405-cd6f-43de-89ad-120b15b6b0ce-ca-trust-extracted\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.756151 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.756128    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51c80405-cd6f-43de-89ad-120b15b6b0ce-image-registry-private-configuration\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.756236 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.756208    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0502af9d-e8c9-4ea7-92a1-c2d11aa10d48-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wfrwl\" (UID: \"0502af9d-e8c9-4ea7-92a1-c2d11aa10d48\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"
Jul 22 12:55:09.756539 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.756517    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51c80405-cd6f-43de-89ad-120b15b6b0ce-installation-pull-secrets\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.756816 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.756795    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51c80405-cd6f-43de-89ad-120b15b6b0ce-registry-tls\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.767220 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.766922    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51c80405-cd6f-43de-89ad-120b15b6b0ce-bound-sa-token\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.767220 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.767052    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"
Jul 22 12:55:09.767497 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.767317    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgq7l\" (UniqueName: \"kubernetes.io/projected/cc75880f-e47d-40b0-aded-055b41bc1a02-kube-api-access-xgq7l\") pod \"downloads-5bc67dcd46-rt7l9\" (UID: \"cc75880f-e47d-40b0-aded-055b41bc1a02\") " pod="openshift-console/downloads-5bc67dcd46-rt7l9"
Jul 22 12:55:09.767915 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.767895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-686nq\" (UniqueName: \"kubernetes.io/projected/51c80405-cd6f-43de-89ad-120b15b6b0ce-kube-api-access-686nq\") pod \"image-registry-7d49b55997-brbvr\" (UID: \"51c80405-cd6f-43de-89ad-120b15b6b0ce\") " pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.857158 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.857107    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-rt7l9"
Jul 22 12:55:09.869602 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.869573    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:09.878174 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.878094    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"
Jul 22 12:55:09.908630 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:09.908522    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt"]
Jul 22 12:55:09.916807 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:09.916779    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34497cf8_da5e_4c4d_b977_32821c24d09e.slice/crio-5df63f1028c296ae630a25d75df157afe9ae9b697323b452685f10f6d1df588d WatchSource:0}: Error finding container 5df63f1028c296ae630a25d75df157afe9ae9b697323b452685f10f6d1df588d: Status 404 returned error can't find the container with id 5df63f1028c296ae630a25d75df157afe9ae9b697323b452685f10f6d1df588d
Jul 22 12:55:10.025541 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.025509    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-rt7l9"]
Jul 22 12:55:10.029260 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:10.029215    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc75880f_e47d_40b0_aded_055b41bc1a02.slice/crio-2157efcb522b0b864704dd3d697ccbcb376a0b2cd01bd3bfe34645dff26f3a57 WatchSource:0}: Error finding container 2157efcb522b0b864704dd3d697ccbcb376a0b2cd01bd3bfe34645dff26f3a57: Status 404 returned error can't find the container with id 2157efcb522b0b864704dd3d697ccbcb376a0b2cd01bd3bfe34645dff26f3a57
Jul 22 12:55:10.051332 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.051303    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d49b55997-brbvr"]
Jul 22 12:55:10.054370 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:10.054345    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51c80405_cd6f_43de_89ad_120b15b6b0ce.slice/crio-d1fee3deb2daa3ffcafe7c4964925be5e04f52c158af81ef67f2e635fe0dba5b WatchSource:0}: Error finding container d1fee3deb2daa3ffcafe7c4964925be5e04f52c158af81ef67f2e635fe0dba5b: Status 404 returned error can't find the container with id d1fee3deb2daa3ffcafe7c4964925be5e04f52c158af81ef67f2e635fe0dba5b
Jul 22 12:55:10.068824 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.068756    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"]
Jul 22 12:55:10.074616 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:10.074558    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0502af9d_e8c9_4ea7_92a1_c2d11aa10d48.slice/crio-7e57068005f58083661d682ad51e82a1837dcf54e2512735fa62bb0c4437ec47 WatchSource:0}: Error finding container 7e57068005f58083661d682ad51e82a1837dcf54e2512735fa62bb0c4437ec47: Status 404 returned error can't find the container with id 7e57068005f58083661d682ad51e82a1837dcf54e2512735fa62bb0c4437ec47
Jul 22 12:55:10.357317 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.357234    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl" event={"ID":"0502af9d-e8c9-4ea7-92a1-c2d11aa10d48","Type":"ContainerStarted","Data":"7e57068005f58083661d682ad51e82a1837dcf54e2512735fa62bb0c4437ec47"}
Jul 22 12:55:10.359031 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.358999    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d49b55997-brbvr" event={"ID":"51c80405-cd6f-43de-89ad-120b15b6b0ce","Type":"ContainerStarted","Data":"b3d2ccbf22a5a22eb01b48f42d8daa366b2dcb7f0167468c41558725c2d32cea"}
Jul 22 12:55:10.359158 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.359051    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d49b55997-brbvr" event={"ID":"51c80405-cd6f-43de-89ad-120b15b6b0ce","Type":"ContainerStarted","Data":"d1fee3deb2daa3ffcafe7c4964925be5e04f52c158af81ef67f2e635fe0dba5b"}
Jul 22 12:55:10.359158 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.359079    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:10.360287 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.360261    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-rt7l9" event={"ID":"cc75880f-e47d-40b0-aded-055b41bc1a02","Type":"ContainerStarted","Data":"2157efcb522b0b864704dd3d697ccbcb376a0b2cd01bd3bfe34645dff26f3a57"}
Jul 22 12:55:10.361530 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.361503    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt" event={"ID":"34497cf8-da5e-4c4d-b977-32821c24d09e","Type":"ContainerStarted","Data":"5df63f1028c296ae630a25d75df157afe9ae9b697323b452685f10f6d1df588d"}
Jul 22 12:55:10.379865 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:10.379809    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d49b55997-brbvr" podStartSLOduration=1.379790338 podStartE2EDuration="1.379790338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:55:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:10.379124306 +0000 UTC m=+62.671105339" watchObservedRunningTime="2026-07-22 12:55:10.379790338 +0000 UTC m=+62.671771370"
Jul 22 12:55:12.343762 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:12.343077    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9p9bf"
Jul 22 12:55:12.374567 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:12.374528    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dh668" event={"ID":"6ae71a63-bc40-4528-8ce5-626c8d3daf02","Type":"ContainerStarted","Data":"4fb865c840e80f32c9daa1089660a1666296b45fb76d7f897d5b8b9f1456a16b"}
Jul 22 12:55:12.403484 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:12.397371    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dh668" podStartSLOduration=17.212198869 podStartE2EDuration="29.397347743s" totalImagesPullingTime="12.185148874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:12.396016425 +0000 UTC m=+64.687997486" watchObservedRunningTime="2026-07-22 12:55:12.397347743 +0000 UTC m=+64.689328777"
Jul 22 12:55:14.385015 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.384913    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl" event={"ID":"0502af9d-e8c9-4ea7-92a1-c2d11aa10d48","Type":"ContainerStarted","Data":"42fc16014d189ea1f030d7fc6b44301662c45330e8b2c684d7f2c25eff014393"}
Jul 22 12:55:14.385425 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.385376    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"
Jul 22 12:55:14.391875 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.391846    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl"
Jul 22 12:55:14.402051 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.401992    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wfrwl" podStartSLOduration=1.384861264 podStartE2EDuration="5.40197133s" totalImagesPullingTime="4.017110066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:14.400559429 +0000 UTC m=+66.692540463" watchObservedRunningTime="2026-07-22 12:55:14.40197133 +0000 UTC m=+66.693952364"
Jul 22 12:55:14.700151 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.700118    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:55:14.700332 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.700174    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:55:14.700332 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.700271    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:55:14.702580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.702531    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 12:55:14.702700 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.702635    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 12:55:14.702700 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.702664    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 12:55:14.713548 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.713523    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 12:55:14.713742 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.713719    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2436034c-df7c-4212-b9d8-b85737f6c858-original-pull-secret\") pod \"global-pull-secret-syncer-5cxpk\" (UID: \"2436034c-df7c-4212-b9d8-b85737f6c858\") " pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:55:14.713742 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.713737    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b615de2-b59e-417c-95d5-42cb2c7fd256-metrics-certs\") pod \"network-metrics-daemon-mncs6\" (UID: \"5b615de2-b59e-417c-95d5-42cb2c7fd256\") " pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:55:14.724210 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.724188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66qtv\" (UniqueName: \"kubernetes.io/projected/e1b75105-2d2e-486a-b65d-2e0242c535b9-kube-api-access-66qtv\") pod \"network-check-target-v25hs\" (UID: \"e1b75105-2d2e-486a-b65d-2e0242c535b9\") " pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:55:14.954398 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.954303    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5cxpk"
Jul 22 12:55:14.964108 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.964077    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fsqrc\""
Jul 22 12:55:14.969225 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.969203    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-frnxs\""
Jul 22 12:55:14.972117 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.972096    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mncs6"
Jul 22 12:55:14.978044 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:14.978013    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:55:15.014021 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.013972    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"]
Jul 22 12:55:15.031592 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.031551    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"]
Jul 22 12:55:15.031747 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.031679    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.034996 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.034897    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 22 12:55:15.035145 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.034899    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 12:55:15.035587 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.035261    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8cpnt\""
Jul 22 12:55:15.035587 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.035292    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 22 12:55:15.035587 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.035516    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 12:55:15.036183 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.036076    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 12:55:15.103763 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.103489    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.103763 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.103548    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.103763 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.103597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6grsh\" (UniqueName: \"kubernetes.io/projected/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-kube-api-access-6grsh\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.103763 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.103638    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.139184 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.138900    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5cxpk"]
Jul 22 12:55:15.143481 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:15.143449    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2436034c_df7c_4212_b9d8_b85737f6c858.slice/crio-f88c6c93e38d9fa100b90770d69c7a3fec64dcd6589c703ae929d74518137799 WatchSource:0}: Error finding container f88c6c93e38d9fa100b90770d69c7a3fec64dcd6589c703ae929d74518137799: Status 404 returned error can't find the container with id f88c6c93e38d9fa100b90770d69c7a3fec64dcd6589c703ae929d74518137799
Jul 22 12:55:15.164099 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.164054    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mncs6"]
Jul 22 12:55:15.168574 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:15.168543    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b615de2_b59e_417c_95d5_42cb2c7fd256.slice/crio-80106ac3c27db66a3b233a386768e5416b90946a5822fbb3bb6ff8a8ccc99a29 WatchSource:0}: Error finding container 80106ac3c27db66a3b233a386768e5416b90946a5822fbb3bb6ff8a8ccc99a29: Status 404 returned error can't find the container with id 80106ac3c27db66a3b233a386768e5416b90946a5822fbb3bb6ff8a8ccc99a29
Jul 22 12:55:15.204807 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.204734    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.204807 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.204803    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.205037 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.204840    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.205037 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:55:15.204874    2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 22 12:55:15.205037 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:55:15.204967    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-prometheus-operator-tls podName:f8017fde-4dbe-4e1b-8ba6-d12d2098df9d nodeName:}" failed. No retries permitted until 2026-07-22 12:55:15.704922428 +0000 UTC m=+67.996903464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-fxp7m" (UID: "f8017fde-4dbe-4e1b-8ba6-d12d2098df9d") : secret "prometheus-operator-tls" not found
Jul 22 12:55:15.205037 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.204881    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6grsh\" (UniqueName: \"kubernetes.io/projected/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-kube-api-access-6grsh\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.205605 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.205588    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.207535 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.207512    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.215264 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.215230    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v25hs"]
Jul 22 12:55:15.218656 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:15.218620    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b75105_2d2e_486a_b65d_2e0242c535b9.slice/crio-447d1c4b33de62bd9d7f05b795f9b3de32286f9474688e468e27934610ded0a4 WatchSource:0}: Error finding container 447d1c4b33de62bd9d7f05b795f9b3de32286f9474688e468e27934610ded0a4: Status 404 returned error can't find the container with id 447d1c4b33de62bd9d7f05b795f9b3de32286f9474688e468e27934610ded0a4
Jul 22 12:55:15.219338 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.219310    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6grsh\" (UniqueName: \"kubernetes.io/projected/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-kube-api-access-6grsh\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.389670 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.389624    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mncs6" event={"ID":"5b615de2-b59e-417c-95d5-42cb2c7fd256","Type":"ContainerStarted","Data":"80106ac3c27db66a3b233a386768e5416b90946a5822fbb3bb6ff8a8ccc99a29"}
Jul 22 12:55:15.391059 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.391032    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5cxpk" event={"ID":"2436034c-df7c-4212-b9d8-b85737f6c858","Type":"ContainerStarted","Data":"f88c6c93e38d9fa100b90770d69c7a3fec64dcd6589c703ae929d74518137799"}
Jul 22 12:55:15.392676 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.392646    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt" event={"ID":"34497cf8-da5e-4c4d-b977-32821c24d09e","Type":"ContainerStarted","Data":"edaf85a9e66c6db58e8d84c35135752c8aa523c38265dd7727b5741f7376b84f"}
Jul 22 12:55:15.393792 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.393771    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v25hs" event={"ID":"e1b75105-2d2e-486a-b65d-2e0242c535b9","Type":"ContainerStarted","Data":"447d1c4b33de62bd9d7f05b795f9b3de32286f9474688e468e27934610ded0a4"}
Jul 22 12:55:15.409035 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.408969    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9586d7b8-gwnrt" podStartSLOduration=1.771465448 podStartE2EDuration="6.40894865s" totalImagesPullingTime="4.637483202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:15.408487146 +0000 UTC m=+67.700468179" watchObservedRunningTime="2026-07-22 12:55:15.40894865 +0000 UTC m=+67.700929692"
Jul 22 12:55:15.709838 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.709797    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.712527 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.712492    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8017fde-4dbe-4e1b-8ba6-d12d2098df9d-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-fxp7m\" (UID: \"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:15.945753 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:15.945721    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"
Jul 22 12:55:16.283802 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:16.283698    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m"]
Jul 22 12:55:16.286912 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:16.286880    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8017fde_4dbe_4e1b_8ba6_d12d2098df9d.slice/crio-63abdb1e8e539b6181c50cfb7054680d130124611bdb6492d61ad6d71db5ba4d WatchSource:0}: Error finding container 63abdb1e8e539b6181c50cfb7054680d130124611bdb6492d61ad6d71db5ba4d: Status 404 returned error can't find the container with id 63abdb1e8e539b6181c50cfb7054680d130124611bdb6492d61ad6d71db5ba4d
Jul 22 12:55:16.400619 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:16.400570    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m" event={"ID":"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d","Type":"ContainerStarted","Data":"63abdb1e8e539b6181c50cfb7054680d130124611bdb6492d61ad6d71db5ba4d"}
Jul 22 12:55:16.403715 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:16.403326    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mncs6" event={"ID":"5b615de2-b59e-417c-95d5-42cb2c7fd256","Type":"ContainerStarted","Data":"f9dc310429d21dc3db24405bfb7225bfe6234ae6f37653af260cb9c7d82284d6"}
Jul 22 12:55:17.411676 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:17.411636    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mncs6" event={"ID":"5b615de2-b59e-417c-95d5-42cb2c7fd256","Type":"ContainerStarted","Data":"38bbd4dc79d24bf5df0e5f1a9113766db02f8135a1a40dae5b1aa555e0729fb4"}
Jul 22 12:55:17.434502 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:17.434221    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mncs6" podStartSLOduration=67.416679286 podStartE2EDuration="1m8.434200472s" totalImagesPullingTime="1.017521186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:17.432906261 +0000 UTC m=+69.724887296" watchObservedRunningTime="2026-07-22 12:55:17.434200472 +0000 UTC m=+69.726181506"
Jul 22 12:55:18.419161 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:18.419000    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m" event={"ID":"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d","Type":"ContainerStarted","Data":"2577036e71e7cb4621798679fac963a2ea4cf7dba168539b5ad81af8a46cdf39"}
Jul 22 12:55:18.419161 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:18.419059    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m" event={"ID":"f8017fde-4dbe-4e1b-8ba6-d12d2098df9d","Type":"ContainerStarted","Data":"790dc75f2afd9c77caef54599c53e57c070cff9804334427a275e5e177216e6f"}
Jul 22 12:55:18.439657 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:18.439540    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-fxp7m" podStartSLOduration=2.5639067989999997 podStartE2EDuration="4.439525287s" totalImagesPullingTime="1.875618488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:18.438393022 +0000 UTC m=+70.730374072" watchObservedRunningTime="2026-07-22 12:55:18.439525287 +0000 UTC m=+70.731506320"
Jul 22 12:55:20.435036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.433595    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"]
Jul 22 12:55:20.439343 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.437581    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.442186 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.441999    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 22 12:55:20.444115 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.443374    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 22 12:55:20.444115 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.443605    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ln62j\""
Jul 22 12:55:20.444115 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.443816    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 22 12:55:20.453669 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.453539    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"]
Jul 22 12:55:20.454702 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.454141    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8t48c"]
Jul 22 12:55:20.458267 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.458207    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.462018 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.461995    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k8hfc\""
Jul 22 12:55:20.462220 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.462204    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 12:55:20.462419 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.462404    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 12:55:20.462663 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.462647    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 12:55:20.551251 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.551002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzx5c\" (UniqueName: \"kubernetes.io/projected/b6404337-0437-4dad-97d6-52ee8e19ca04-kube-api-access-xzx5c\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.551251 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.551053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8488358c-f988-4072-8d9b-be6e9f10a0ba-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.551251 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.551083    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8488358c-f988-4072-8d9b-be6e9f10a0ba-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.551251 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.551108    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6404337-0437-4dad-97d6-52ee8e19ca04-sys\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.551251 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.551143    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6404337-0437-4dad-97d6-52ee8e19ca04-metrics-client-ca\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.552010 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.551795    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-tls\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.552010 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.551847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.552010 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.551888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdln2\" (UniqueName: \"kubernetes.io/projected/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-api-access-tdln2\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.552010 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.551915    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.552731 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.552473    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.552731 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.552547    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-accelerators-collector-config\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.552731 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.552573    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.552731 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.552607    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6404337-0437-4dad-97d6-52ee8e19ca04-root\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.552731 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.552629    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-wtmp\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.552731 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.552655    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-textfile\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.653809 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.653778    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-tls\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.653988 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.653826    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.653988 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.653864    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdln2\" (UniqueName: \"kubernetes.io/projected/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-api-access-tdln2\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.653988 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.653889    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.653988 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.653917    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.653988 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.653975    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-accelerators-collector-config\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.654244 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654002    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.654244 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6404337-0437-4dad-97d6-52ee8e19ca04-root\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.654244 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-wtmp\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.654244 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-textfile\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.654244 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzx5c\" (UniqueName: \"kubernetes.io/projected/b6404337-0437-4dad-97d6-52ee8e19ca04-kube-api-access-xzx5c\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.654244 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654147    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8488358c-f988-4072-8d9b-be6e9f10a0ba-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.654244 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654176    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8488358c-f988-4072-8d9b-be6e9f10a0ba-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.654244 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654197    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6404337-0437-4dad-97d6-52ee8e19ca04-sys\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.654244 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654230    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6404337-0437-4dad-97d6-52ee8e19ca04-metrics-client-ca\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.655542 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654827    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6404337-0437-4dad-97d6-52ee8e19ca04-metrics-client-ca\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.655542 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.654900    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6404337-0437-4dad-97d6-52ee8e19ca04-root\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.655542 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.655116    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-wtmp\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.655542 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.655343    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-textfile\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.656036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.655884    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.656036 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.656001    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6404337-0437-4dad-97d6-52ee8e19ca04-sys\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.656185 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:55:20.656099    2575 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 22 12:55:20.656185 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.656103    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8488358c-f988-4072-8d9b-be6e9f10a0ba-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.656185 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:55:20.656157    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-tls podName:8488358c-f988-4072-8d9b-be6e9f10a0ba nodeName:}" failed. No retries permitted until 2026-07-22 12:55:21.1561374 +0000 UTC m=+73.448118415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-5dpk9" (UID: "8488358c-f988-4072-8d9b-be6e9f10a0ba") : secret "kube-state-metrics-tls" not found
Jul 22 12:55:20.656387 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.656190    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8488358c-f988-4072-8d9b-be6e9f10a0ba-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.656571 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.656526    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-accelerators-collector-config\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.660105 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.660083    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.661515 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.660882    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-tls\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.661515 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.661438    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6404337-0437-4dad-97d6-52ee8e19ca04-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.667834 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.667771    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdln2\" (UniqueName: \"kubernetes.io/projected/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-api-access-tdln2\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:20.671098 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.671071    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzx5c\" (UniqueName: \"kubernetes.io/projected/b6404337-0437-4dad-97d6-52ee8e19ca04-kube-api-access-xzx5c\") pod \"node-exporter-8t48c\" (UID: \"b6404337-0437-4dad-97d6-52ee8e19ca04\") " pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:20.777397 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:20.777309    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8t48c"
Jul 22 12:55:21.159272 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:21.159233    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:21.162152 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:21.162126    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8488358c-f988-4072-8d9b-be6e9f10a0ba-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-5dpk9\" (UID: \"8488358c-f988-4072-8d9b-be6e9f10a0ba\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:21.356845 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:21.356813    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"
Jul 22 12:55:22.083540 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:22.083505    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6404337_0437_4dad_97d6_52ee8e19ca04.slice/crio-7183f75d00d2d73d5da3cf81f74a63bb2bebdbd1ca21a6d915a31d3f3f9888a9 WatchSource:0}: Error finding container 7183f75d00d2d73d5da3cf81f74a63bb2bebdbd1ca21a6d915a31d3f3f9888a9: Status 404 returned error can't find the container with id 7183f75d00d2d73d5da3cf81f74a63bb2bebdbd1ca21a6d915a31d3f3f9888a9
Jul 22 12:55:22.235916 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.235890    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-5dpk9"]
Jul 22 12:55:22.239591 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:22.239570    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8488358c_f988_4072_8d9b_be6e9f10a0ba.slice/crio-07ca9d64bcf6f4842218791ad28862bc9224f9ebdf62337a5f75d98ea830f315 WatchSource:0}: Error finding container 07ca9d64bcf6f4842218791ad28862bc9224f9ebdf62337a5f75d98ea830f315: Status 404 returned error can't find the container with id 07ca9d64bcf6f4842218791ad28862bc9224f9ebdf62337a5f75d98ea830f315
Jul 22 12:55:22.402778 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.402741    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-f597666cd-d9n4j"]
Jul 22 12:55:22.406013 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.405985    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.408991 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.408848    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 22 12:55:22.408991 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.408875    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4fxnc\""
Jul 22 12:55:22.409193 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.408955    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 22 12:55:22.409251 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.409192    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 22 12:55:22.409302 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.409195    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6nl3t9hjscrcp\""
Jul 22 12:55:22.409644 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.409625    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 22 12:55:22.409909 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.409647    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 22 12:55:22.423402 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.423329    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f597666cd-d9n4j"]
Jul 22 12:55:22.443758 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.443680    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v25hs" event={"ID":"e1b75105-2d2e-486a-b65d-2e0242c535b9","Type":"ContainerStarted","Data":"ddca26dad6ede6c0af8e528497c972b8807716902e045f90853d24232d71d6b7"}
Jul 22 12:55:22.443758 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.443733    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:55:22.446273 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.446245    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5cxpk" event={"ID":"2436034c-df7c-4212-b9d8-b85737f6c858","Type":"ContainerStarted","Data":"8bf08cda2239ac9dbbc7199db5a773a011c2e4c838b0905e08a9910a08762227"}
Jul 22 12:55:22.447516 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.447490    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8t48c" event={"ID":"b6404337-0437-4dad-97d6-52ee8e19ca04","Type":"ContainerStarted","Data":"7183f75d00d2d73d5da3cf81f74a63bb2bebdbd1ca21a6d915a31d3f3f9888a9"}
Jul 22 12:55:22.448997 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.448943    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9" event={"ID":"8488358c-f988-4072-8d9b-be6e9f10a0ba","Type":"ContainerStarted","Data":"07ca9d64bcf6f4842218791ad28862bc9224f9ebdf62337a5f75d98ea830f315"}
Jul 22 12:55:22.464015 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.463971    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v25hs" podStartSLOduration=66.602497868 podStartE2EDuration="1m13.463951135s" totalImagesPullingTime="6.861453267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:22.463012944 +0000 UTC m=+74.754993976" watchObservedRunningTime="2026-07-22 12:55:22.463951135 +0000 UTC m=+74.755932166"
Jul 22 12:55:22.471495 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.471470    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.471619 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.471511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44420be8-bc55-4236-a425-eae4067ef8e3-metrics-client-ca\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.471619 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.471538    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.471619 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.471564    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rhbn\" (UniqueName: \"kubernetes.io/projected/44420be8-bc55-4236-a425-eae4067ef8e3-kube-api-access-8rhbn\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.471803 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.471642    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.471803 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.471677    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.471803 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.471721    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-tls\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.471803 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.471790    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-grpc-tls\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.481254 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.481211    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5cxpk" podStartSLOduration=66.533379523 podStartE2EDuration="1m13.481196238s" totalImagesPullingTime="6.947816715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:22.47978128 +0000 UTC m=+74.771762311" watchObservedRunningTime="2026-07-22 12:55:22.481196238 +0000 UTC m=+74.773177270"
Jul 22 12:55:22.572718 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.572668    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.572914 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.572731    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.572914 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.572830    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-tls\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.572914 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.572869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-grpc-tls\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.573100 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.572913    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.573100 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.572962    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44420be8-bc55-4236-a425-eae4067ef8e3-metrics-client-ca\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.573100 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.572990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.573100 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.573016    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rhbn\" (UniqueName: \"kubernetes.io/projected/44420be8-bc55-4236-a425-eae4067ef8e3-kube-api-access-8rhbn\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.573869 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.573799    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44420be8-bc55-4236-a425-eae4067ef8e3-metrics-client-ca\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.575794 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.575751    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.575890 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.575790    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.576276 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.576254    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-grpc-tls\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.576387 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.576295    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.576467 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.576399    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-tls\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.577256 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.577177    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44420be8-bc55-4236-a425-eae4067ef8e3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.583011 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.582987    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rhbn\" (UniqueName: \"kubernetes.io/projected/44420be8-bc55-4236-a425-eae4067ef8e3-kube-api-access-8rhbn\") pod \"thanos-querier-f597666cd-d9n4j\" (UID: \"44420be8-bc55-4236-a425-eae4067ef8e3\") " pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.719023 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.718908    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:22.885592 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:22.885560    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f597666cd-d9n4j"]
Jul 22 12:55:22.890173 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:22.890138    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44420be8_bc55_4236_a425_eae4067ef8e3.slice/crio-690dab9d53d064c2e23811b53cc63187581d32b04eba4bee5f04118416493cdd WatchSource:0}: Error finding container 690dab9d53d064c2e23811b53cc63187581d32b04eba4bee5f04118416493cdd: Status 404 returned error can't find the container with id 690dab9d53d064c2e23811b53cc63187581d32b04eba4bee5f04118416493cdd
Jul 22 12:55:23.454258 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:23.454220    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j" event={"ID":"44420be8-bc55-4236-a425-eae4067ef8e3","Type":"ContainerStarted","Data":"690dab9d53d064c2e23811b53cc63187581d32b04eba4bee5f04118416493cdd"}
Jul 22 12:55:25.146777 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.146740    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"]
Jul 22 12:55:25.149203 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.149181    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:25.151768 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.151742    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-m8cbw\""
Jul 22 12:55:25.151973 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.151741    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 22 12:55:25.159626 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.159605    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"]
Jul 22 12:55:25.196686 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.196650    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsn8g\" (UniqueName: \"kubernetes.io/projected/bd71ac03-0e60-4347-9e79-253b9281f35c-kube-api-access-bsn8g\") pod \"monitoring-plugin-7b7b7bc8bf-qvcvc\" (UID: \"bd71ac03-0e60-4347-9e79-253b9281f35c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:25.196864 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.196789    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd71ac03-0e60-4347-9e79-253b9281f35c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qvcvc\" (UID: \"bd71ac03-0e60-4347-9e79-253b9281f35c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:25.297264 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.297226    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd71ac03-0e60-4347-9e79-253b9281f35c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qvcvc\" (UID: \"bd71ac03-0e60-4347-9e79-253b9281f35c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:25.297440 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.297287    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsn8g\" (UniqueName: \"kubernetes.io/projected/bd71ac03-0e60-4347-9e79-253b9281f35c-kube-api-access-bsn8g\") pod \"monitoring-plugin-7b7b7bc8bf-qvcvc\" (UID: \"bd71ac03-0e60-4347-9e79-253b9281f35c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:25.297440 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:55:25.297405    2575 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 22 12:55:25.297533 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:55:25.297486    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd71ac03-0e60-4347-9e79-253b9281f35c-monitoring-plugin-cert podName:bd71ac03-0e60-4347-9e79-253b9281f35c nodeName:}" failed. No retries permitted until 2026-07-22 12:55:25.797464578 +0000 UTC m=+78.089445599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/bd71ac03-0e60-4347-9e79-253b9281f35c-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-qvcvc" (UID: "bd71ac03-0e60-4347-9e79-253b9281f35c") : secret "monitoring-plugin-cert" not found
Jul 22 12:55:25.309512 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.309208    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsn8g\" (UniqueName: \"kubernetes.io/projected/bd71ac03-0e60-4347-9e79-253b9281f35c-kube-api-access-bsn8g\") pod \"monitoring-plugin-7b7b7bc8bf-qvcvc\" (UID: \"bd71ac03-0e60-4347-9e79-253b9281f35c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:25.801997 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.801955    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd71ac03-0e60-4347-9e79-253b9281f35c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qvcvc\" (UID: \"bd71ac03-0e60-4347-9e79-253b9281f35c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:25.804775 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:25.804747    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bd71ac03-0e60-4347-9e79-253b9281f35c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qvcvc\" (UID: \"bd71ac03-0e60-4347-9e79-253b9281f35c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:26.060804 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.060717    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:26.654371 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.652253    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:55:26.697474 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.697435    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:55:26.697672 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.697574    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.701569 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.700420    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 12:55:26.701569 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.700672    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 12:55:26.701569 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.700906    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 12:55:26.702524 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.702114    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 12:55:26.702524 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.702242    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 12:55:26.702524 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.702288    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 12:55:26.702524 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.702342    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 12:55:26.702524 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.702352    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 12:55:26.702524 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.702401    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 12:55:26.702909 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.702600    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 12:55:26.702909 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.702767    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 12:55:26.702909 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.702893    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 12:55:26.703077 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.703065    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d5en6gdbtc399\""
Jul 22 12:55:26.703319 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.703298    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 12:55:26.703552 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.703526    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kh6ql\""
Jul 22 12:55:26.705074 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.705056    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 12:55:26.708836 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.708815    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 12:55:26.812176 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812143    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812399 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812192    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812399 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812269    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-config-out\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812399 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812330    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-config\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812399 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812631 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812455    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812631 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812482    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812631 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812510    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812631 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812631 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812558    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87zhd\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-kube-api-access-87zhd\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812631 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812608    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812901 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812645    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812901 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812672    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812901 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812698    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812901 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812901 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812754    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-web-config\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812901 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812776    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812901 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812806    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.812901 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812887    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.813328 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.812940    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.914967 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.914851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.914967 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.914910    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.914967 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.914947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915255 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.914972    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915255 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.914993    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915255 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915014    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87zhd\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-kube-api-access-87zhd\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915255 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915058    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915255 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915081    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915255 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915107    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915255 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915255 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915141    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-web-config\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915336    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915394    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915422    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915487    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915524    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-config-out\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.915621 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.915548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-config\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.916160 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.916079    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.916219 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.916184    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.916706 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.916680    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.916867 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.916846    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.919395 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.917244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.919395 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.917869    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.919395 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.918556    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.919395 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.919073    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.919395 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.919332    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.920333 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.920241    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.920465 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.920330    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.921228 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.921188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.921743 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.921718    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-config\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.921828 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.921777    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.922019 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.921996    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.922100 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.922073    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.922724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.922684    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.923058 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.923011    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-config-out\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.925879 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.925835    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-web-config\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:26.927124 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:26.927105    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87zhd\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-kube-api-access-87zhd\") pod \"prometheus-k8s-0\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:27.012678 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:27.012645    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:31.370667 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:31.370558    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d49b55997-brbvr"
Jul 22 12:55:31.798322 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:31.798298    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:55:31.801474 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:31.801440    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29c6df1_c9ca_4d24_8485_a56768efdd52.slice/crio-3e192e47b420e0af6c29241ff71636ef86c55c7a9c8f0405dbc0a04645cc809c WatchSource:0}: Error finding container 3e192e47b420e0af6c29241ff71636ef86c55c7a9c8f0405dbc0a04645cc809c: Status 404 returned error can't find the container with id 3e192e47b420e0af6c29241ff71636ef86c55c7a9c8f0405dbc0a04645cc809c
Jul 22 12:55:31.822023 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:31.819020    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"]
Jul 22 12:55:31.827209 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:55:31.827177    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd71ac03_0e60_4347_9e79_253b9281f35c.slice/crio-589fac90b959b6c624365661e24e1de0deaebad2f3240ca4e9d263c057c3c6a9 WatchSource:0}: Error finding container 589fac90b959b6c624365661e24e1de0deaebad2f3240ca4e9d263c057c3c6a9: Status 404 returned error can't find the container with id 589fac90b959b6c624365661e24e1de0deaebad2f3240ca4e9d263c057c3c6a9
Jul 22 12:55:32.498984 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:32.498919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-rt7l9" event={"ID":"cc75880f-e47d-40b0-aded-055b41bc1a02","Type":"ContainerStarted","Data":"3f78e992432cce89e1b526d6583ae324e833c5cd9241bd85b5c7292ddd6e8e74"}
Jul 22 12:55:32.499490 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:32.499458    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-rt7l9"
Jul 22 12:55:32.501950 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:32.501899    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8t48c" event={"ID":"b6404337-0437-4dad-97d6-52ee8e19ca04","Type":"ContainerStarted","Data":"d148579326df3f757c45f3621aa409ad505192e54cc3271ef469f58b4532523f"}
Jul 22 12:55:32.503808 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:32.503778    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerStarted","Data":"3e192e47b420e0af6c29241ff71636ef86c55c7a9c8f0405dbc0a04645cc809c"}
Jul 22 12:55:32.505453 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:32.505430    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc" event={"ID":"bd71ac03-0e60-4347-9e79-253b9281f35c","Type":"ContainerStarted","Data":"589fac90b959b6c624365661e24e1de0deaebad2f3240ca4e9d263c057c3c6a9"}
Jul 22 12:55:32.508773 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:32.508673    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9" event={"ID":"8488358c-f988-4072-8d9b-be6e9f10a0ba","Type":"ContainerStarted","Data":"02a44264c617d60c53dd03bfbfef552fb6a69d96e54a474075fcb5512ae7a626"}
Jul 22 12:55:32.508773 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:32.508726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9" event={"ID":"8488358c-f988-4072-8d9b-be6e9f10a0ba","Type":"ContainerStarted","Data":"ae9e2c04b7f9d9bc5478f57a3e376d9da2b6ba3b572ca04366337047c71aa1ca"}
Jul 22 12:55:32.517118 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:32.516490    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-rt7l9" podStartSLOduration=1.90622166 podStartE2EDuration="23.516471278s" totalImagesPullingTime="21.610249618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:32.514880944 +0000 UTC m=+84.806861997" watchObservedRunningTime="2026-07-22 12:55:32.516471278 +0000 UTC m=+84.808452317"
Jul 22 12:55:32.521239 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:32.521216    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-rt7l9"
Jul 22 12:55:33.517451 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:33.517370    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9" event={"ID":"8488358c-f988-4072-8d9b-be6e9f10a0ba","Type":"ContainerStarted","Data":"d01d941648cd18b819ebec64da923c0aea3283a315aafcc38fd38616c5ec929d"}
Jul 22 12:55:33.520826 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:33.520274    2575 generic.go:356] "Generic (PLEG): container finished" podID="b6404337-0437-4dad-97d6-52ee8e19ca04" containerID="d148579326df3f757c45f3621aa409ad505192e54cc3271ef469f58b4532523f" exitCode=0
Jul 22 12:55:33.520826 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:33.520370    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8t48c" event={"ID":"b6404337-0437-4dad-97d6-52ee8e19ca04","Type":"ContainerDied","Data":"d148579326df3f757c45f3621aa409ad505192e54cc3271ef469f58b4532523f"}
Jul 22 12:55:33.524695 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:33.524431    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j" event={"ID":"44420be8-bc55-4236-a425-eae4067ef8e3","Type":"ContainerStarted","Data":"2a0308b992bcda0cf81afab7aca4e90863fe508a7b72274d45fc744456e412bb"}
Jul 22 12:55:33.524695 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:33.524516    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j" event={"ID":"44420be8-bc55-4236-a425-eae4067ef8e3","Type":"ContainerStarted","Data":"f7ec55cbf7db36acb47b5f673cc5530fc76417a8663652b8dd67f461cdf8561a"}
Jul 22 12:55:33.524695 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:33.524558    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j" event={"ID":"44420be8-bc55-4236-a425-eae4067ef8e3","Type":"ContainerStarted","Data":"ef3fc29f2cd1b1ad7ce278d51e872490f935d66172adff523adf41a36cca94ef"}
Jul 22 12:55:33.538063 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:33.538011    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-5dpk9" podStartSLOduration=4.181973751 podStartE2EDuration="13.537992644s" totalImagesPullingTime="9.356018893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:33.535838431 +0000 UTC m=+85.827819490" watchObservedRunningTime="2026-07-22 12:55:33.537992644 +0000 UTC m=+85.829973677"
Jul 22 12:55:35.537439 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:35.537356    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc" event={"ID":"bd71ac03-0e60-4347-9e79-253b9281f35c","Type":"ContainerStarted","Data":"6489557636a3f12e35d65ecb187316695b450c68faa79adfd2621b904c6d2122"}
Jul 22 12:55:35.537883 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:35.537718    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:35.540844 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:35.540763    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8t48c" event={"ID":"b6404337-0437-4dad-97d6-52ee8e19ca04","Type":"ContainerStarted","Data":"c4d4f675cf32ae3307b2fb6ea5f1e23ee3a1805b931156b21d48373b0d864c65"}
Jul 22 12:55:35.540844 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:35.540812    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8t48c" event={"ID":"b6404337-0437-4dad-97d6-52ee8e19ca04","Type":"ContainerStarted","Data":"5a264976aef47c52899025fb3b0e83aa509f8c1a24f1383ce6f44dd7a8c738a6"}
Jul 22 12:55:35.542633 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:35.542602    2575 generic.go:356] "Generic (PLEG): container finished" podID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerID="9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f" exitCode=0
Jul 22 12:55:35.542757 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:35.542653    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerDied","Data":"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"}
Jul 22 12:55:35.544085 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:35.544064    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc"
Jul 22 12:55:35.555648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:35.555604    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qvcvc" podStartSLOduration=7.489030932 podStartE2EDuration="10.555590526s" totalImagesPullingTime="3.066559594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:35.553920535 +0000 UTC m=+87.845901567" watchObservedRunningTime="2026-07-22 12:55:35.555590526 +0000 UTC m=+87.847571578"
Jul 22 12:55:35.575384 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:35.575325    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8t48c" podStartSLOduration=5.944776852 podStartE2EDuration="15.575308689s" totalImagesPullingTime="9.513789852s" totalInitContainerRuntime="116.741985ms" isStatefulPod=true podCreationTimestamp="2026-07-22 12:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:35.573512118 +0000 UTC m=+87.865493151" watchObservedRunningTime="2026-07-22 12:55:35.575308689 +0000 UTC m=+87.867289720"
Jul 22 12:55:36.553117 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:36.552623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j" event={"ID":"44420be8-bc55-4236-a425-eae4067ef8e3","Type":"ContainerStarted","Data":"7571b1bca18f5c0d4bbc9d451c8a3112cb82b2eda7f01ea58608ea142b87a8b2"}
Jul 22 12:55:36.553117 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:36.552851    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j" event={"ID":"44420be8-bc55-4236-a425-eae4067ef8e3","Type":"ContainerStarted","Data":"890f2055273c050e9abaef321dc6763d064fe7580dc68ea0cae0fd9a328759a1"}
Jul 22 12:55:36.553117 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:36.552910    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j" event={"ID":"44420be8-bc55-4236-a425-eae4067ef8e3","Type":"ContainerStarted","Data":"de8dd0001614b4b202d8383ecf51816a53d9f5af0022aedf4d31b51af905be90"}
Jul 22 12:55:36.554023 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:36.553955    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:36.582741 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:36.582673    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j" podStartSLOduration=2.260451995 podStartE2EDuration="14.582648935s" totalImagesPullingTime="12.32219694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:55:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:36.579921625 +0000 UTC m=+88.871902683" watchObservedRunningTime="2026-07-22 12:55:36.582648935 +0000 UTC m=+88.874629960"
Jul 22 12:55:38.570750 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:38.570717    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-f597666cd-d9n4j"
Jul 22 12:55:40.580231 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:40.580146    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerStarted","Data":"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"}
Jul 22 12:55:40.580612 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:40.580256    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerStarted","Data":"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"}
Jul 22 12:55:40.580612 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:40.580295    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerStarted","Data":"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"}
Jul 22 12:55:40.580612 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:40.580321    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerStarted","Data":"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"}
Jul 22 12:55:41.587891 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:41.587854    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerStarted","Data":"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"}
Jul 22 12:55:41.588391 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:41.587921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerStarted","Data":"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"}
Jul 22 12:55:41.619089 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:41.619032    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=8.231698156 podStartE2EDuration="15.619011931s" totalImagesPullingTime="7.359550326s" totalInitContainerRuntime="27.763449ms" isStatefulPod=false podCreationTimestamp="2026-07-22 12:55:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:55:41.617107562 +0000 UTC m=+93.909088623" watchObservedRunningTime="2026-07-22 12:55:41.619011931 +0000 UTC m=+93.910992963"
Jul 22 12:55:42.013198 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:42.013108    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:55:53.456701 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:55:53.456667    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v25hs"
Jul 22 12:56:27.013276 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:27.013239    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:27.032336 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:27.032309    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:27.794625 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:27.794598    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:45.057846 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.057806    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:56:45.058388 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.058144    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="prometheus" containerID="cri-o://fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d" gracePeriod=600
Jul 22 12:56:45.058388 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.058189    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="thanos-sidecar" containerID="cri-o://f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19" gracePeriod=600
Jul 22 12:56:45.058388 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.058208    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="config-reloader" containerID="cri-o://cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8" gracePeriod=600
Jul 22 12:56:45.058388 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.058230    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy-web" containerID="cri-o://a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c" gracePeriod=600
Jul 22 12:56:45.058388 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.058265    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb" gracePeriod=600
Jul 22 12:56:45.058657 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.058389    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy" containerID="cri-o://fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0" gracePeriod=600
Jul 22 12:56:45.316074 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.316006    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:45.494179 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494150    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-tls\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494364 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494186    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-kubelet-serving-ca-bundle\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494364 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494217    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87zhd\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-kube-api-access-87zhd\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494364 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494240    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-db\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494364 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494267    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-config-out\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494364 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494294    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-serving-certs-ca-bundle\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494364 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494321    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-2\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494364 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494346    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-trusted-ca-bundle\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494375    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494417    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-1\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494447    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-web-config\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494485    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-0\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494516    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-config\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494545    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-metrics-client-certs\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494573    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-kube-rbac-proxy\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494596    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-grpc-tls\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494611    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494632    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494656    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-thanos-prometheus-http-client-file\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494689    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-tls-assets\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.494724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494712    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:56:45.495429 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.494742    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-metrics-client-ca\") pod \"e29c6df1-c9ca-4d24-8485-a56768efdd52\" (UID: \"e29c6df1-c9ca-4d24-8485-a56768efdd52\") "
Jul 22 12:56:45.495429 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.495037    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.495429 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.495058    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.495429 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.495263    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:56:45.495429 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.495377    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:56:45.495773 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.495498    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:56:45.495773 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.495508    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:56:45.497906 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.497875    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:56:45.498066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.497880    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:56:45.498066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.497977    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:56:45.498066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.497983    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-config-out" (OuterVolumeSpecName: "config-out") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:56:45.498066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.498026    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:56:45.498429 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.498381    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:56:45.498522 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.498438    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-kube-api-access-87zhd" (OuterVolumeSpecName: "kube-api-access-87zhd") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "kube-api-access-87zhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:56:45.499137 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.499108    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:56:45.499287 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.499236    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:56:45.499546 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.499523    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:56:45.499633 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.499545    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-config" (OuterVolumeSpecName: "config") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:56:45.499698 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.499681    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:56:45.500022 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.499999    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:56:45.508362 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.508341    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-web-config" (OuterVolumeSpecName: "web-config") pod "e29c6df1-c9ca-4d24-8485-a56768efdd52" (UID: "e29c6df1-c9ca-4d24-8485-a56768efdd52"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:56:45.596378 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596329    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-configmap-metrics-client-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596378 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596372    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596378 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596384    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87zhd\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-kube-api-access-87zhd\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596378 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596393    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-db\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596401    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e29c6df1-c9ca-4d24-8485-a56768efdd52-config-out\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596410    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596418    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596429    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596437    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596445    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-web-config\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596453    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e29c6df1-c9ca-4d24-8485-a56768efdd52-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596471    2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-config\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596481    2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-metrics-client-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596489    2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-kube-rbac-proxy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596497    2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-grpc-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596506    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596515    2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e29c6df1-c9ca-4d24-8485-a56768efdd52-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.596648 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.596524    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e29c6df1-c9ca-4d24-8485-a56768efdd52-tls-assets\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 12:56:45.853066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.852982    2575 generic.go:356] "Generic (PLEG): container finished" podID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerID="1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb" exitCode=0
Jul 22 12:56:45.853066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853006    2575 generic.go:356] "Generic (PLEG): container finished" podID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerID="fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0" exitCode=0
Jul 22 12:56:45.853066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853012    2575 generic.go:356] "Generic (PLEG): container finished" podID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerID="a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c" exitCode=0
Jul 22 12:56:45.853066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853020    2575 generic.go:356] "Generic (PLEG): container finished" podID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerID="f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19" exitCode=0
Jul 22 12:56:45.853066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853025    2575 generic.go:356] "Generic (PLEG): container finished" podID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerID="cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8" exitCode=0
Jul 22 12:56:45.853066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853032    2575 generic.go:356] "Generic (PLEG): container finished" podID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerID="fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d" exitCode=0
Jul 22 12:56:45.853066 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853059    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:45.853377 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerDied","Data":"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"}
Jul 22 12:56:45.853377 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853212    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerDied","Data":"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"}
Jul 22 12:56:45.853377 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853266    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerDied","Data":"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"}
Jul 22 12:56:45.853377 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853276    2575 scope.go:117] "RemoveContainer" containerID="1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"
Jul 22 12:56:45.853377 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853299    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerDied","Data":"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"}
Jul 22 12:56:45.853377 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853332    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerDied","Data":"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"}
Jul 22 12:56:45.853377 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853358    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerDied","Data":"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"}
Jul 22 12:56:45.853686 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.853390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e29c6df1-c9ca-4d24-8485-a56768efdd52","Type":"ContainerDied","Data":"3e192e47b420e0af6c29241ff71636ef86c55c7a9c8f0405dbc0a04645cc809c"}
Jul 22 12:56:45.861733 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.861713    2575 scope.go:117] "RemoveContainer" containerID="fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"
Jul 22 12:56:45.868554 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.868540    2575 scope.go:117] "RemoveContainer" containerID="a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"
Jul 22 12:56:45.875233 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.875216    2575 scope.go:117] "RemoveContainer" containerID="f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"
Jul 22 12:56:45.880724 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.880703    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:56:45.882511 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.882489    2575 scope.go:117] "RemoveContainer" containerID="cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"
Jul 22 12:56:45.885920 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.885897    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:56:45.889410 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.889397    2575 scope.go:117] "RemoveContainer" containerID="fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"
Jul 22 12:56:45.896359 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.896340    2575 scope.go:117] "RemoveContainer" containerID="9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"
Jul 22 12:56:45.902591 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.902574    2575 scope.go:117] "RemoveContainer" containerID="1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"
Jul 22 12:56:45.902851 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:56:45.902832    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": container with ID starting with 1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb not found: ID does not exist" containerID="1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"
Jul 22 12:56:45.902897 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.902863    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"} err="failed to get container status \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": rpc error: code = NotFound desc = could not find container \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": container with ID starting with 1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb not found: ID does not exist"
Jul 22 12:56:45.902897 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.902896    2575 scope.go:117] "RemoveContainer" containerID="fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"
Jul 22 12:56:45.903150 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:56:45.903133    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": container with ID starting with fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0 not found: ID does not exist" containerID="fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"
Jul 22 12:56:45.903206 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.903157    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"} err="failed to get container status \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": rpc error: code = NotFound desc = could not find container \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": container with ID starting with fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0 not found: ID does not exist"
Jul 22 12:56:45.903206 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.903173    2575 scope.go:117] "RemoveContainer" containerID="a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"
Jul 22 12:56:45.903353 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:56:45.903333    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": container with ID starting with a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c not found: ID does not exist" containerID="a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"
Jul 22 12:56:45.903391 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.903358    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"} err="failed to get container status \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": rpc error: code = NotFound desc = could not find container \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": container with ID starting with a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c not found: ID does not exist"
Jul 22 12:56:45.903391 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.903373    2575 scope.go:117] "RemoveContainer" containerID="f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"
Jul 22 12:56:45.903554 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:56:45.903539    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": container with ID starting with f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19 not found: ID does not exist" containerID="f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"
Jul 22 12:56:45.903590 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.903558    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"} err="failed to get container status \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": rpc error: code = NotFound desc = could not find container \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": container with ID starting with f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19 not found: ID does not exist"
Jul 22 12:56:45.903590 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.903570    2575 scope.go:117] "RemoveContainer" containerID="cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"
Jul 22 12:56:45.903809 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:56:45.903794    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": container with ID starting with cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8 not found: ID does not exist" containerID="cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"
Jul 22 12:56:45.903854 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.903813    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"} err="failed to get container status \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": rpc error: code = NotFound desc = could not find container \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": container with ID starting with cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8 not found: ID does not exist"
Jul 22 12:56:45.903854 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.903830    2575 scope.go:117] "RemoveContainer" containerID="fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"
Jul 22 12:56:45.904096 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:56:45.904077    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": container with ID starting with fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d not found: ID does not exist" containerID="fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"
Jul 22 12:56:45.904167 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.904100    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"} err="failed to get container status \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": rpc error: code = NotFound desc = could not find container \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": container with ID starting with fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d not found: ID does not exist"
Jul 22 12:56:45.904167 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.904116    2575 scope.go:117] "RemoveContainer" containerID="9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"
Jul 22 12:56:45.904352 ip-10-0-140-133 kubenswrapper[2575]: E0722 12:56:45.904337    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": container with ID starting with 9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f not found: ID does not exist" containerID="9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"
Jul 22 12:56:45.904392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.904355    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"} err="failed to get container status \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": rpc error: code = NotFound desc = could not find container \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": container with ID starting with 9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f not found: ID does not exist"
Jul 22 12:56:45.904392 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.904368    2575 scope.go:117] "RemoveContainer" containerID="1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"
Jul 22 12:56:45.904599 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.904579    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"} err="failed to get container status \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": rpc error: code = NotFound desc = could not find container \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": container with ID starting with 1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb not found: ID does not exist"
Jul 22 12:56:45.904668 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.904600    2575 scope.go:117] "RemoveContainer" containerID="fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"
Jul 22 12:56:45.904858 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.904840    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"} err="failed to get container status \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": rpc error: code = NotFound desc = could not find container \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": container with ID starting with fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0 not found: ID does not exist"
Jul 22 12:56:45.904917 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.904865    2575 scope.go:117] "RemoveContainer" containerID="a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"
Jul 22 12:56:45.905140 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.905119    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"} err="failed to get container status \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": rpc error: code = NotFound desc = could not find container \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": container with ID starting with a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c not found: ID does not exist"
Jul 22 12:56:45.905202 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.905138    2575 scope.go:117] "RemoveContainer" containerID="f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"
Jul 22 12:56:45.905417 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.905399    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"} err="failed to get container status \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": rpc error: code = NotFound desc = could not find container \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": container with ID starting with f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19 not found: ID does not exist"
Jul 22 12:56:45.905474 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.905426    2575 scope.go:117] "RemoveContainer" containerID="cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"
Jul 22 12:56:45.905728 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.905699    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"} err="failed to get container status \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": rpc error: code = NotFound desc = could not find container \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": container with ID starting with cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8 not found: ID does not exist"
Jul 22 12:56:45.905728 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.905727    2575 scope.go:117] "RemoveContainer" containerID="fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"
Jul 22 12:56:45.905990 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.905973    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"} err="failed to get container status \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": rpc error: code = NotFound desc = could not find container \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": container with ID starting with fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d not found: ID does not exist"
Jul 22 12:56:45.906043 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.905991    2575 scope.go:117] "RemoveContainer" containerID="9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"
Jul 22 12:56:45.906203 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.906182    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"} err="failed to get container status \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": rpc error: code = NotFound desc = could not find container \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": container with ID starting with 9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f not found: ID does not exist"
Jul 22 12:56:45.906239 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.906205    2575 scope.go:117] "RemoveContainer" containerID="1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"
Jul 22 12:56:45.906439 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.906421    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"} err="failed to get container status \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": rpc error: code = NotFound desc = could not find container \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": container with ID starting with 1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb not found: ID does not exist"
Jul 22 12:56:45.906487 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.906439    2575 scope.go:117] "RemoveContainer" containerID="fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"
Jul 22 12:56:45.906626 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.906611    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"} err="failed to get container status \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": rpc error: code = NotFound desc = could not find container \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": container with ID starting with fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0 not found: ID does not exist"
Jul 22 12:56:45.906670 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.906627    2575 scope.go:117] "RemoveContainer" containerID="a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"
Jul 22 12:56:45.906850 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.906833    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"} err="failed to get container status \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": rpc error: code = NotFound desc = could not find container \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": container with ID starting with a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c not found: ID does not exist"
Jul 22 12:56:45.906891 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.906851    2575 scope.go:117] "RemoveContainer" containerID="f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"
Jul 22 12:56:45.907114 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907094    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"} err="failed to get container status \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": rpc error: code = NotFound desc = could not find container \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": container with ID starting with f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19 not found: ID does not exist"
Jul 22 12:56:45.907163 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907114    2575 scope.go:117] "RemoveContainer" containerID="cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"
Jul 22 12:56:45.907308 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907292    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"} err="failed to get container status \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": rpc error: code = NotFound desc = could not find container \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": container with ID starting with cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8 not found: ID does not exist"
Jul 22 12:56:45.907357 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907307    2575 scope.go:117] "RemoveContainer" containerID="fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"
Jul 22 12:56:45.907518 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907504    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"} err="failed to get container status \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": rpc error: code = NotFound desc = could not find container \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": container with ID starting with fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d not found: ID does not exist"
Jul 22 12:56:45.907556 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907519    2575 scope.go:117] "RemoveContainer" containerID="9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"
Jul 22 12:56:45.907708 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907693    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"} err="failed to get container status \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": rpc error: code = NotFound desc = could not find container \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": container with ID starting with 9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f not found: ID does not exist"
Jul 22 12:56:45.907744 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907708    2575 scope.go:117] "RemoveContainer" containerID="1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"
Jul 22 12:56:45.907890 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907874    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"} err="failed to get container status \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": rpc error: code = NotFound desc = could not find container \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": container with ID starting with 1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb not found: ID does not exist"
Jul 22 12:56:45.907947 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.907892    2575 scope.go:117] "RemoveContainer" containerID="fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"
Jul 22 12:56:45.908088 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908073    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"} err="failed to get container status \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": rpc error: code = NotFound desc = could not find container \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": container with ID starting with fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0 not found: ID does not exist"
Jul 22 12:56:45.908129 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908090    2575 scope.go:117] "RemoveContainer" containerID="a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"
Jul 22 12:56:45.908302 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908279    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"} err="failed to get container status \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": rpc error: code = NotFound desc = could not find container \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": container with ID starting with a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c not found: ID does not exist"
Jul 22 12:56:45.908353 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908302    2575 scope.go:117] "RemoveContainer" containerID="f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"
Jul 22 12:56:45.908473 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908458    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"} err="failed to get container status \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": rpc error: code = NotFound desc = could not find container \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": container with ID starting with f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19 not found: ID does not exist"
Jul 22 12:56:45.908510 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908475    2575 scope.go:117] "RemoveContainer" containerID="cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"
Jul 22 12:56:45.908658 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908643    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"} err="failed to get container status \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": rpc error: code = NotFound desc = could not find container \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": container with ID starting with cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8 not found: ID does not exist"
Jul 22 12:56:45.908698 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908658    2575 scope.go:117] "RemoveContainer" containerID="fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"
Jul 22 12:56:45.908846 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908831    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"} err="failed to get container status \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": rpc error: code = NotFound desc = could not find container \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": container with ID starting with fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d not found: ID does not exist"
Jul 22 12:56:45.908886 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.908846    2575 scope.go:117] "RemoveContainer" containerID="9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"
Jul 22 12:56:45.909070 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909047    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"} err="failed to get container status \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": rpc error: code = NotFound desc = could not find container \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": container with ID starting with 9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f not found: ID does not exist"
Jul 22 12:56:45.909070 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909069    2575 scope.go:117] "RemoveContainer" containerID="1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"
Jul 22 12:56:45.909297 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909277    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"} err="failed to get container status \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": rpc error: code = NotFound desc = could not find container \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": container with ID starting with 1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb not found: ID does not exist"
Jul 22 12:56:45.909338 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909298    2575 scope.go:117] "RemoveContainer" containerID="fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"
Jul 22 12:56:45.909545 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909528    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"} err="failed to get container status \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": rpc error: code = NotFound desc = could not find container \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": container with ID starting with fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0 not found: ID does not exist"
Jul 22 12:56:45.909587 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909546    2575 scope.go:117] "RemoveContainer" containerID="a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"
Jul 22 12:56:45.909753 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909737    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"} err="failed to get container status \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": rpc error: code = NotFound desc = could not find container \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": container with ID starting with a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c not found: ID does not exist"
Jul 22 12:56:45.909802 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909753    2575 scope.go:117] "RemoveContainer" containerID="f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"
Jul 22 12:56:45.909969 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909951    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"} err="failed to get container status \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": rpc error: code = NotFound desc = could not find container \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": container with ID starting with f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19 not found: ID does not exist"
Jul 22 12:56:45.910015 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.909970    2575 scope.go:117] "RemoveContainer" containerID="cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"
Jul 22 12:56:45.910188 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.910170    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"} err="failed to get container status \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": rpc error: code = NotFound desc = could not find container \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": container with ID starting with cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8 not found: ID does not exist"
Jul 22 12:56:45.910188 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.910186    2575 scope.go:117] "RemoveContainer" containerID="fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"
Jul 22 12:56:45.910399 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.910381    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"} err="failed to get container status \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": rpc error: code = NotFound desc = could not find container \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": container with ID starting with fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d not found: ID does not exist"
Jul 22 12:56:45.910446 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.910399    2575 scope.go:117] "RemoveContainer" containerID="9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"
Jul 22 12:56:45.910613 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.910594    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"} err="failed to get container status \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": rpc error: code = NotFound desc = could not find container \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": container with ID starting with 9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f not found: ID does not exist"
Jul 22 12:56:45.910683 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.910615    2575 scope.go:117] "RemoveContainer" containerID="1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"
Jul 22 12:56:45.910818 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.910803    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb"} err="failed to get container status \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": rpc error: code = NotFound desc = could not find container \"1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb\": container with ID starting with 1e6a9de62bec44ec792c68b309ffab00e2bd7d4b53eea0ae9695a115b1a8dccb not found: ID does not exist"
Jul 22 12:56:45.910867 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.910818    2575 scope.go:117] "RemoveContainer" containerID="fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"
Jul 22 12:56:45.911012 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.910995    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0"} err="failed to get container status \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": rpc error: code = NotFound desc = could not find container \"fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0\": container with ID starting with fd219ed1cdb5fdcaa439841f5167cf3b5161805f421251d27530c097d281d4a0 not found: ID does not exist"
Jul 22 12:56:45.911063 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911013    2575 scope.go:117] "RemoveContainer" containerID="a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"
Jul 22 12:56:45.911190 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911174    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c"} err="failed to get container status \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": rpc error: code = NotFound desc = could not find container \"a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c\": container with ID starting with a42914fea16bd15eabbed51724a5e12c3a7de5443920b0d8a39d08ae460ef67c not found: ID does not exist"
Jul 22 12:56:45.911256 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911193    2575 scope.go:117] "RemoveContainer" containerID="f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"
Jul 22 12:56:45.911367 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911352    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19"} err="failed to get container status \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": rpc error: code = NotFound desc = could not find container \"f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19\": container with ID starting with f8072e793844adc689b8f3a38faf53101e5a8b0e61497377fff3d1c932e7ea19 not found: ID does not exist"
Jul 22 12:56:45.911406 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911367    2575 scope.go:117] "RemoveContainer" containerID="cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"
Jul 22 12:56:45.911523 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911506    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8"} err="failed to get container status \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": rpc error: code = NotFound desc = could not find container \"cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8\": container with ID starting with cef71c602acfcb199d1486e3cd0ead4ca0ae73a9c1497f637cafd0854f312ba8 not found: ID does not exist"
Jul 22 12:56:45.911585 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911524    2575 scope.go:117] "RemoveContainer" containerID="fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"
Jul 22 12:56:45.911763 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911746    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d"} err="failed to get container status \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": rpc error: code = NotFound desc = could not find container \"fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d\": container with ID starting with fb962d91482e84b53dc077e3309c2c1a7d72bc3a51f0db4358f07131dfb5767d not found: ID does not exist"
Jul 22 12:56:45.911799 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911773    2575 scope.go:117] "RemoveContainer" containerID="9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"
Jul 22 12:56:45.912005 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.911988    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f"} err="failed to get container status \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": rpc error: code = NotFound desc = could not find container \"9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f\": container with ID starting with 9e914f269e65c19ad8ad0e5dbaecf4ad35f3cc991ea254d6c9230e5a065d970f not found: ID does not exist"
Jul 22 12:56:45.922748 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.922729    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:56:45.923029 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923016    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy"
Jul 22 12:56:45.923078 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923030    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy"
Jul 22 12:56:45.923078 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923040    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="prometheus"
Jul 22 12:56:45.923078 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923046    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="prometheus"
Jul 22 12:56:45.923078 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923053    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="thanos-sidecar"
Jul 22 12:56:45.923078 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923059    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="thanos-sidecar"
Jul 22 12:56:45.923078 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923072    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy-web"
Jul 22 12:56:45.923078 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923078    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy-web"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923089    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="init-config-reloader"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923095    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="init-config-reloader"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923127    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy-thanos"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923132    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy-thanos"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923145    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="config-reloader"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923150    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="config-reloader"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923195    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy-thanos"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923204    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy-web"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923209    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="kube-rbac-proxy"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923216    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="prometheus"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923221    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="thanos-sidecar"
Jul 22 12:56:45.923268 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.923227    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" containerName="config-reloader"
Jul 22 12:56:45.927250 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.927236    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:45.932606 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.931511    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 12:56:45.932952 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.932913    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 12:56:45.933081 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.933024    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 12:56:45.933169 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.933083    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d5en6gdbtc399\""
Jul 22 12:56:45.938453 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.938436    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 12:56:45.947010 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.946990    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 12:56:45.947100 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.946997    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 12:56:45.947182 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.947164    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 12:56:45.947474 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.947460    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 12:56:45.947811 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.947795    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 12:56:45.947866 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.947798    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 12:56:45.949043 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.948565    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 12:56:45.949043 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.948693    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kh6ql\""
Jul 22 12:56:45.949201 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.949070    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 12:56:45.949252 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.949225    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 12:56:45.988383 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.988346    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 12:56:45.991516 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.991490    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:56:45.991719 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:45.991702    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 12:56:46.101009 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.100961    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101022    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101051    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrj52\" (UniqueName: \"kubernetes.io/projected/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-kube-api-access-xrj52\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101073    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101095    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-web-config\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101154    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101210    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101235    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101258    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101297    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101328    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-config-out\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101368 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-config\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101793 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101793 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101448    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101793 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101495    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101793 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101512    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101793 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101539    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.101793 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.101559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.202951 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.202837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-config\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.202951 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.202872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.202951 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.202894    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.202951 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.202913    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.202951 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.202956    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.202976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.202994    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203028    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203063    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203088    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrj52\" (UniqueName: \"kubernetes.io/projected/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-kube-api-access-xrj52\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203115    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203140    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-web-config\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203196    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203218    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203256    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203279    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203313 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203947 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203352    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203947 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203380    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-config-out\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203947 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203947 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203656    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.203947 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.203841    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.204431 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.204408    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.204580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.204465    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.205774 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.205746    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.205882 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.205863    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.206330 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.206305    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.206631 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.206608    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-config-out\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.206717 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.206663    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-config\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.207009 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.206968    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.207636 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.207594    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.207995 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.207969    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.208087 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.207997    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.208087 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.208052    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.208230 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.208212    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-web-config\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.209046 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.209028    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.209174 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.209156    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.209241 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.209161    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.213944 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.213901    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrj52\" (UniqueName: \"kubernetes.io/projected/eeb39e7e-95b3-40cc-8e3a-c81d36dc3315-kube-api-access-xrj52\") pod \"prometheus-k8s-0\" (UID: \"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.237015 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.236992    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:56:46.372576 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.372401    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 12:56:46.375105 ip-10-0-140-133 kubenswrapper[2575]: W0722 12:56:46.375082    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb39e7e_95b3_40cc_8e3a_c81d36dc3315.slice/crio-c8120dd9a01ad58fa64c2a6fae23e052aec9e0051561bec859d4d119a918b305 WatchSource:0}: Error finding container c8120dd9a01ad58fa64c2a6fae23e052aec9e0051561bec859d4d119a918b305: Status 404 returned error can't find the container with id c8120dd9a01ad58fa64c2a6fae23e052aec9e0051561bec859d4d119a918b305
Jul 22 12:56:46.860073 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.860044    2575 generic.go:356] "Generic (PLEG): container finished" podID="eeb39e7e-95b3-40cc-8e3a-c81d36dc3315" containerID="c8269fe0241b410a032076718e32ae6863fc7fee9151a7b8da11c33c154d706b" exitCode=0
Jul 22 12:56:46.860248 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.860123    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315","Type":"ContainerDied","Data":"c8269fe0241b410a032076718e32ae6863fc7fee9151a7b8da11c33c154d706b"}
Jul 22 12:56:46.860248 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.860180    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315","Type":"ContainerStarted","Data":"c8120dd9a01ad58fa64c2a6fae23e052aec9e0051561bec859d4d119a918b305"}
Jul 22 12:56:46.948039 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:46.948013    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e29c6df1-c9ca-4d24-8485-a56768efdd52" path="/var/lib/kubelet/pods/e29c6df1-c9ca-4d24-8485-a56768efdd52/volumes"
Jul 22 12:56:47.874084 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:47.874048    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315","Type":"ContainerStarted","Data":"2ae14902992db86ce2da1b404a9d755b4481afafa937b5696ee67501ac282964"}
Jul 22 12:56:47.874580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:47.874163    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315","Type":"ContainerStarted","Data":"27fb934a1751bbe6a3443fa7d4310b5afc632bde16e9a089bbbbf225cb930b1b"}
Jul 22 12:56:47.874580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:47.874207    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315","Type":"ContainerStarted","Data":"1c6b15e717747b01bdf3e5aad9b9dc491513caeeccc2f0927d33d2be3a5dab1b"}
Jul 22 12:56:47.874580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:47.874256    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315","Type":"ContainerStarted","Data":"a22041a0a97da4af0601d95495fac5c1121fa2ec3021fb1cc209986588fe86fb"}
Jul 22 12:56:47.874580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:47.874295    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315","Type":"ContainerStarted","Data":"e5c801336559c15f36396c033bf03705c150dbfd2b94eb8582247f98a36669fb"}
Jul 22 12:56:47.874580 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:47.874335    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"eeb39e7e-95b3-40cc-8e3a-c81d36dc3315","Type":"ContainerStarted","Data":"d30361ead66a1f7c51b2b8c11bd45e815dbd04f622c8dcadb125dd771169d674"}
Jul 22 12:56:47.904178 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:47.904115    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.886577329 podStartE2EDuration="2.904087973s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.510644ms" isStatefulPod=false podCreationTimestamp="2026-07-22 12:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:56:47.902584521 +0000 UTC m=+160.194565555" watchObservedRunningTime="2026-07-22 12:56:47.904087973 +0000 UTC m=+160.196069007"
Jul 22 12:56:51.237464 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:56:51.237426    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:57:46.237463 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:57:46.237430    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:57:46.252432 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:57:46.252406    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:57:47.114357 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:57:47.114331    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 12:59:08.784372 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:59:08.784345    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 12:59:08.789697 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:59:08.789675    2575 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 12:59:08.790380 ip-10-0-140-133 kubenswrapper[2575]: I0722 12:59:08.790358    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:00:08.705519 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.705443    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"]
Jul 22 13:00:08.707791 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.707775    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.714496 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.714474    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 13:00:08.715082 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.715066    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 13:00:08.715189 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.715171    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 13:00:08.715348 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.715328    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2qg5c\""
Jul 22 13:00:08.715606 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.715593    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 13:00:08.742370 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.742340    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"]
Jul 22 13:00:08.766010 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.765979    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phcwq\" (UniqueName: \"kubernetes.io/projected/d980c01e-8def-4a8e-af30-071333cea555-kube-api-access-phcwq\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.766168 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.766037    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d980c01e-8def-4a8e-af30-071333cea555-webhook-cert\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.766168 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.766067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d980c01e-8def-4a8e-af30-071333cea555-manifests\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.766168 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.766099    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d980c01e-8def-4a8e-af30-071333cea555-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.766168 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.766136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d980c01e-8def-4a8e-af30-071333cea555-tmp\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.867387 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.867354    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d980c01e-8def-4a8e-af30-071333cea555-tmp\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.867569 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.867407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phcwq\" (UniqueName: \"kubernetes.io/projected/d980c01e-8def-4a8e-af30-071333cea555-kube-api-access-phcwq\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.867569 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.867465    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d980c01e-8def-4a8e-af30-071333cea555-webhook-cert\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.867569 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.867507    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d980c01e-8def-4a8e-af30-071333cea555-manifests\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.867569 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.867539    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d980c01e-8def-4a8e-af30-071333cea555-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.867859 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.867839    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d980c01e-8def-4a8e-af30-071333cea555-manifests\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.869648 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.869630    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d980c01e-8def-4a8e-af30-071333cea555-tmp\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.871813 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.871798    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 13:00:08.880725 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.880698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d980c01e-8def-4a8e-af30-071333cea555-webhook-cert\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.880844 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.880728    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d980c01e-8def-4a8e-af30-071333cea555-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:08.885426 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.885410    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 13:00:08.896190 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.896169    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 13:00:08.906506 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:08.906481    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phcwq\" (UniqueName: \"kubernetes.io/projected/d980c01e-8def-4a8e-af30-071333cea555-kube-api-access-phcwq\") pod \"opendatahub-operator-controller-manager-74989f8477-n87vv\" (UID: \"d980c01e-8def-4a8e-af30-071333cea555\") " pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:09.020541 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:09.020461    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2qg5c\""
Jul 22 13:00:09.027923 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:09.027902    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:09.160579 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:09.160556    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"]
Jul 22 13:00:09.162839 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:00:09.162813    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd980c01e_8def_4a8e_af30_071333cea555.slice/crio-530501b3793b52f799ebfb0aabd6588cb7d8de53c11a515b8ac4ad3617775024 WatchSource:0}: Error finding container 530501b3793b52f799ebfb0aabd6588cb7d8de53c11a515b8ac4ad3617775024: Status 404 returned error can't find the container with id 530501b3793b52f799ebfb0aabd6588cb7d8de53c11a515b8ac4ad3617775024
Jul 22 13:00:09.164643 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:09.164628    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 13:00:09.648058 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:09.648027    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv" event={"ID":"d980c01e-8def-4a8e-af30-071333cea555","Type":"ContainerStarted","Data":"530501b3793b52f799ebfb0aabd6588cb7d8de53c11a515b8ac4ad3617775024"}
Jul 22 13:00:12.664724 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:12.664682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv" event={"ID":"d980c01e-8def-4a8e-af30-071333cea555","Type":"ContainerStarted","Data":"4a8fc1ed5666120a9a3bbb1a4163e074e9d7feb24c66b6a5a59559d6806b1f26"}
Jul 22 13:00:13.671200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:13.671115    2575 generic.go:356] "Generic (PLEG): container finished" podID="d980c01e-8def-4a8e-af30-071333cea555" containerID="4a8fc1ed5666120a9a3bbb1a4163e074e9d7feb24c66b6a5a59559d6806b1f26" exitCode=0
Jul 22 13:00:13.671200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:13.671159    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv" event={"ID":"d980c01e-8def-4a8e-af30-071333cea555","Type":"ContainerDied","Data":"4a8fc1ed5666120a9a3bbb1a4163e074e9d7feb24c66b6a5a59559d6806b1f26"}
Jul 22 13:00:14.677976 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:14.677921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv" event={"ID":"d980c01e-8def-4a8e-af30-071333cea555","Type":"ContainerStarted","Data":"0af881587f9971135572fd50cd03ef9d55a60b0be169677c9c935744519babc9"}
Jul 22 13:00:14.678364 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:14.678090    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:14.713099 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:14.713047    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv" podStartSLOduration=2.219063353 podStartE2EDuration="6.713030424s" totalImagesPullingTime="3.421558542s" totalInitContainerRuntime="1.072408529s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:00:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:00:14.711523834 +0000 UTC m=+367.003504911" watchObservedRunningTime="2026-07-22 13:00:14.713030424 +0000 UTC m=+367.005011462"
Jul 22 13:00:25.684696 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:25.684663    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74989f8477-n87vv"
Jul 22 13:00:37.648011 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.647972    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"]
Jul 22 13:00:37.651551 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.651532    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.655504 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.655484    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 22 13:00:37.655504 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.655493    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 22 13:00:37.655697 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.655484    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w49jz\""
Jul 22 13:00:37.655697 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.655518    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 13:00:37.655697 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.655526    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 22 13:00:37.655697 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.655530    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 13:00:37.662185 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.662163    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"]
Jul 22 13:00:37.718904 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.718872    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71770cb3-0614-4602-aa05-300f5419bd28-cert\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.718904 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.718911    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/71770cb3-0614-4602-aa05-300f5419bd28-manager-config\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.719129 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.718955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlg6w\" (UniqueName: \"kubernetes.io/projected/71770cb3-0614-4602-aa05-300f5419bd28-kube-api-access-rlg6w\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.719129 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.719028    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/71770cb3-0614-4602-aa05-300f5419bd28-metrics-cert\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.820033 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.819987    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71770cb3-0614-4602-aa05-300f5419bd28-cert\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.820224 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.820047    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/71770cb3-0614-4602-aa05-300f5419bd28-manager-config\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.820224 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.820085    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlg6w\" (UniqueName: \"kubernetes.io/projected/71770cb3-0614-4602-aa05-300f5419bd28-kube-api-access-rlg6w\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.820500 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.820472    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/71770cb3-0614-4602-aa05-300f5419bd28-metrics-cert\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.820898 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.820837    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/71770cb3-0614-4602-aa05-300f5419bd28-manager-config\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.823395 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.823308    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71770cb3-0614-4602-aa05-300f5419bd28-cert\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.823753 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.823731    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/71770cb3-0614-4602-aa05-300f5419bd28-metrics-cert\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.830789 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.830766    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlg6w\" (UniqueName: \"kubernetes.io/projected/71770cb3-0614-4602-aa05-300f5419bd28-kube-api-access-rlg6w\") pod \"lws-controller-manager-68759d94bb-vjdjn\" (UID: \"71770cb3-0614-4602-aa05-300f5419bd28\") " pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:37.961670 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:37.961582    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:38.106944 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:38.106897    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"]
Jul 22 13:00:38.109356 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:00:38.109328    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71770cb3_0614_4602_aa05_300f5419bd28.slice/crio-b9cdf73375f0b1ca62a36be1d22a1d3e8fda145626a24df57960182ddb748984 WatchSource:0}: Error finding container b9cdf73375f0b1ca62a36be1d22a1d3e8fda145626a24df57960182ddb748984: Status 404 returned error can't find the container with id b9cdf73375f0b1ca62a36be1d22a1d3e8fda145626a24df57960182ddb748984
Jul 22 13:00:38.787723 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:38.787689    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn" event={"ID":"71770cb3-0614-4602-aa05-300f5419bd28","Type":"ContainerStarted","Data":"b9cdf73375f0b1ca62a36be1d22a1d3e8fda145626a24df57960182ddb748984"}
Jul 22 13:00:40.386895 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.386856    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"]
Jul 22 13:00:40.392988 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.392960    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.396372 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.396313    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 13:00:40.396751 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.396309    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 13:00:40.396900 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.396701    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 13:00:40.397640 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.397329    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zgjth\""
Jul 22 13:00:40.397640 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.397442    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 13:00:40.400404 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.400082    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"]
Jul 22 13:00:40.568451 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.568416    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqr94\" (UniqueName: \"kubernetes.io/projected/fda46392-a2b7-46cc-828f-03f688ebd9ea-kube-api-access-kqr94\") pod \"kube-auth-proxy-767b4d54f8-5r995\" (UID: \"fda46392-a2b7-46cc-828f-03f688ebd9ea\") " pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.568625 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.568475    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fda46392-a2b7-46cc-828f-03f688ebd9ea-tmp\") pod \"kube-auth-proxy-767b4d54f8-5r995\" (UID: \"fda46392-a2b7-46cc-828f-03f688ebd9ea\") " pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.568625 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.568532    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda46392-a2b7-46cc-828f-03f688ebd9ea-tls-certs\") pod \"kube-auth-proxy-767b4d54f8-5r995\" (UID: \"fda46392-a2b7-46cc-828f-03f688ebd9ea\") " pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.669347 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.669258    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda46392-a2b7-46cc-828f-03f688ebd9ea-tls-certs\") pod \"kube-auth-proxy-767b4d54f8-5r995\" (UID: \"fda46392-a2b7-46cc-828f-03f688ebd9ea\") " pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.669347 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.669338    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqr94\" (UniqueName: \"kubernetes.io/projected/fda46392-a2b7-46cc-828f-03f688ebd9ea-kube-api-access-kqr94\") pod \"kube-auth-proxy-767b4d54f8-5r995\" (UID: \"fda46392-a2b7-46cc-828f-03f688ebd9ea\") " pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.669598 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.669385    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fda46392-a2b7-46cc-828f-03f688ebd9ea-tmp\") pod \"kube-auth-proxy-767b4d54f8-5r995\" (UID: \"fda46392-a2b7-46cc-828f-03f688ebd9ea\") " pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.671798 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.671772    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fda46392-a2b7-46cc-828f-03f688ebd9ea-tmp\") pod \"kube-auth-proxy-767b4d54f8-5r995\" (UID: \"fda46392-a2b7-46cc-828f-03f688ebd9ea\") " pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.672031 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.672010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda46392-a2b7-46cc-828f-03f688ebd9ea-tls-certs\") pod \"kube-auth-proxy-767b4d54f8-5r995\" (UID: \"fda46392-a2b7-46cc-828f-03f688ebd9ea\") " pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.680440 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.680392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqr94\" (UniqueName: \"kubernetes.io/projected/fda46392-a2b7-46cc-828f-03f688ebd9ea-kube-api-access-kqr94\") pod \"kube-auth-proxy-767b4d54f8-5r995\" (UID: \"fda46392-a2b7-46cc-828f-03f688ebd9ea\") " pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:40.708772 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:40.708744    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"
Jul 22 13:00:41.085445 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:41.085412    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-767b4d54f8-5r995"]
Jul 22 13:00:41.086070 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:00:41.086041    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda46392_a2b7_46cc_828f_03f688ebd9ea.slice/crio-47073da3750010d67a0831a7d183f71972b4e2c838705091113c45013e136249 WatchSource:0}: Error finding container 47073da3750010d67a0831a7d183f71972b4e2c838705091113c45013e136249: Status 404 returned error can't find the container with id 47073da3750010d67a0831a7d183f71972b4e2c838705091113c45013e136249
Jul 22 13:00:41.805659 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:41.805623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995" event={"ID":"fda46392-a2b7-46cc-828f-03f688ebd9ea","Type":"ContainerStarted","Data":"47073da3750010d67a0831a7d183f71972b4e2c838705091113c45013e136249"}
Jul 22 13:00:41.807313 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:41.807281    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn" event={"ID":"71770cb3-0614-4602-aa05-300f5419bd28","Type":"ContainerStarted","Data":"0ae4a6467a5310e3a77e2f20d18a54812b12dc2fcea2c5cc240c275877fee994"}
Jul 22 13:00:41.807447 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:41.807418    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:00:41.853628 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:41.853570    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn" podStartSLOduration=2.003070103 podStartE2EDuration="4.853550307s" totalImagesPullingTime="2.850480204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:00:41.853197515 +0000 UTC m=+394.145178549" watchObservedRunningTime="2026-07-22 13:00:41.853550307 +0000 UTC m=+394.145531340"
Jul 22 13:00:43.822473 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:43.822412    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995" event={"ID":"fda46392-a2b7-46cc-828f-03f688ebd9ea","Type":"ContainerStarted","Data":"db3ac4af06f74b1c2652e7c0cc4f4869ab544720d26f324bccb9667e4da6e1de"}
Jul 22 13:00:43.844655 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:43.844598    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-767b4d54f8-5r995" podStartSLOduration=1.385494434 podStartE2EDuration="3.844581208s" totalImagesPullingTime="2.459086774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:00:43.842305269 +0000 UTC m=+396.134286314" watchObservedRunningTime="2026-07-22 13:00:43.844581208 +0000 UTC m=+396.136562240"
Jul 22 13:00:52.815036 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:00:52.815001    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-68759d94bb-vjdjn"
Jul 22 13:02:12.293030 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.292998    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"]
Jul 22 13:02:12.295326 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.295310    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:12.298377 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.298350    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2lfdr\""
Jul 22 13:02:12.298377 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.298374    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 13:02:12.298564 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.298416    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 13:02:12.312788 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.312760    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"]
Jul 22 13:02:12.327969 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.327921    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sgxn\" (UniqueName: \"kubernetes.io/projected/7754211f-d5d5-4ba4-b63d-c8f726b4b712-kube-api-access-6sgxn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" (UID: \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:12.328142 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.327983    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7754211f-d5d5-4ba4-b63d-c8f726b4b712-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" (UID: \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:12.429341 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.429306    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sgxn\" (UniqueName: \"kubernetes.io/projected/7754211f-d5d5-4ba4-b63d-c8f726b4b712-kube-api-access-6sgxn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" (UID: \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:12.429526 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.429360    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7754211f-d5d5-4ba4-b63d-c8f726b4b712-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" (UID: \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:12.429826 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.429807    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7754211f-d5d5-4ba4-b63d-c8f726b4b712-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" (UID: \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:12.442148 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.442125    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sgxn\" (UniqueName: \"kubernetes.io/projected/7754211f-d5d5-4ba4-b63d-c8f726b4b712-kube-api-access-6sgxn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" (UID: \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:12.605328 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.605292    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:12.746181 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:12.746156    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"]
Jul 22 13:02:12.748226 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:02:12.748202    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7754211f_d5d5_4ba4_b63d_c8f726b4b712.slice/crio-983c167b046dec54bd3cad13c134db00cca010e3b6f8e89c01d239e3e277ca2b WatchSource:0}: Error finding container 983c167b046dec54bd3cad13c134db00cca010e3b6f8e89c01d239e3e277ca2b: Status 404 returned error can't find the container with id 983c167b046dec54bd3cad13c134db00cca010e3b6f8e89c01d239e3e277ca2b
Jul 22 13:02:13.200045 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:13.200012    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp" event={"ID":"7754211f-d5d5-4ba4-b63d-c8f726b4b712","Type":"ContainerStarted","Data":"983c167b046dec54bd3cad13c134db00cca010e3b6f8e89c01d239e3e277ca2b"}
Jul 22 13:02:18.229485 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:18.229452    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp" event={"ID":"7754211f-d5d5-4ba4-b63d-c8f726b4b712","Type":"ContainerStarted","Data":"c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27"}
Jul 22 13:02:18.229955 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:18.229684    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:18.256922 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:18.256874    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp" podStartSLOduration=1.3582137859999999 podStartE2EDuration="6.256858943s" totalImagesPullingTime="4.898645157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:02:18.254869355 +0000 UTC m=+490.546850388" watchObservedRunningTime="2026-07-22 13:02:18.256858943 +0000 UTC m=+490.548839976"
Jul 22 13:02:29.236637 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:29.236609    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:31.205102 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.205061    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"]
Jul 22 13:02:31.205594 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.205289    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" containerName="manager" containerID="cri-o://c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27" gracePeriod=2
Jul 22 13:02:31.208114 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.208073    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"]
Jul 22 13:02:31.208530 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:02:31.208502    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" is forbidden: User \"system:node:ip-10-0-140-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-133.ec2.internal' and this object" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:31.245053 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.245028    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"]
Jul 22 13:02:31.245498 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.245480    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" containerName="manager"
Jul 22 13:02:31.245498 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.245501    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" containerName="manager"
Jul 22 13:02:31.245666 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.245582    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" containerName="manager"
Jul 22 13:02:31.248073 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.248053    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:31.251912 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:02:31.251880    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" is forbidden: User \"system:node:ip-10-0-140-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-133.ec2.internal' and this object" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:31.267171 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.267143    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"]
Jul 22 13:02:31.397526 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.397498    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh7gl\" (UniqueName: \"kubernetes.io/projected/e554cc17-dd34-4183-9605-b87c75c2bff1-kube-api-access-gh7gl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqcvb\" (UID: \"e554cc17-dd34-4183-9605-b87c75c2bff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:31.397662 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.397537    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e554cc17-dd34-4183-9605-b87c75c2bff1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqcvb\" (UID: \"e554cc17-dd34-4183-9605-b87c75c2bff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:31.451802 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.451779    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:31.454150 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:02:31.454124    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" is forbidden: User \"system:node:ip-10-0-140-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-133.ec2.internal' and this object" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:31.498166 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.498082    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh7gl\" (UniqueName: \"kubernetes.io/projected/e554cc17-dd34-4183-9605-b87c75c2bff1-kube-api-access-gh7gl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqcvb\" (UID: \"e554cc17-dd34-4183-9605-b87c75c2bff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:31.498166 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.498125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e554cc17-dd34-4183-9605-b87c75c2bff1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqcvb\" (UID: \"e554cc17-dd34-4183-9605-b87c75c2bff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:31.498446 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.498431    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e554cc17-dd34-4183-9605-b87c75c2bff1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqcvb\" (UID: \"e554cc17-dd34-4183-9605-b87c75c2bff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:31.508566 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.508536    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh7gl\" (UniqueName: \"kubernetes.io/projected/e554cc17-dd34-4183-9605-b87c75c2bff1-kube-api-access-gh7gl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dqcvb\" (UID: \"e554cc17-dd34-4183-9605-b87c75c2bff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:31.585439 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.585395    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:31.599233 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.599208    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7754211f-d5d5-4ba4-b63d-c8f726b4b712-extensions-socket-volume\") pod \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\" (UID: \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\") "
Jul 22 13:02:31.599360 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.599248    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sgxn\" (UniqueName: \"kubernetes.io/projected/7754211f-d5d5-4ba4-b63d-c8f726b4b712-kube-api-access-6sgxn\") pod \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\" (UID: \"7754211f-d5d5-4ba4-b63d-c8f726b4b712\") "
Jul 22 13:02:31.599731 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.599703    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7754211f-d5d5-4ba4-b63d-c8f726b4b712-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7754211f-d5d5-4ba4-b63d-c8f726b4b712" (UID: "7754211f-d5d5-4ba4-b63d-c8f726b4b712"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:31.601532 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.601511    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7754211f-d5d5-4ba4-b63d-c8f726b4b712-kube-api-access-6sgxn" (OuterVolumeSpecName: "kube-api-access-6sgxn") pod "7754211f-d5d5-4ba4-b63d-c8f726b4b712" (UID: "7754211f-d5d5-4ba4-b63d-c8f726b4b712"). InnerVolumeSpecName "kube-api-access-6sgxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:02:31.700599 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.700571    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7754211f-d5d5-4ba4-b63d-c8f726b4b712-extensions-socket-volume\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:02:31.700599 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.700596    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sgxn\" (UniqueName: \"kubernetes.io/projected/7754211f-d5d5-4ba4-b63d-c8f726b4b712-kube-api-access-6sgxn\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:02:31.720116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:31.720094    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"]
Jul 22 13:02:31.722663 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:02:31.722641    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode554cc17_dd34_4183_9605_b87c75c2bff1.slice/crio-06b6a7d639545d7ec5b1108ed1b552885bbd18c2ae17116ed36b1fde90c4daad WatchSource:0}: Error finding container 06b6a7d639545d7ec5b1108ed1b552885bbd18c2ae17116ed36b1fde90c4daad: Status 404 returned error can't find the container with id 06b6a7d639545d7ec5b1108ed1b552885bbd18c2ae17116ed36b1fde90c4daad
Jul 22 13:02:32.294976 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.294911    2575 generic.go:356] "Generic (PLEG): container finished" podID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" containerID="c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27" exitCode=0
Jul 22 13:02:32.295421 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.294993    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:32.295421 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.294983    2575 scope.go:117] "RemoveContainer" containerID="c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27"
Jul 22 13:02:32.296914 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.296886    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb" event={"ID":"e554cc17-dd34-4183-9605-b87c75c2bff1","Type":"ContainerStarted","Data":"36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe"}
Jul 22 13:02:32.297059 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.296986    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb" event={"ID":"e554cc17-dd34-4183-9605-b87c75c2bff1","Type":"ContainerStarted","Data":"06b6a7d639545d7ec5b1108ed1b552885bbd18c2ae17116ed36b1fde90c4daad"}
Jul 22 13:02:32.297059 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.297017    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:32.298110 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:02:32.298089    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" is forbidden: User \"system:node:ip-10-0-140-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-133.ec2.internal' and this object" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:32.300282 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:02:32.300250    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" is forbidden: User \"system:node:ip-10-0-140-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-133.ec2.internal' and this object" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:32.304590 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.304574    2575 scope.go:117] "RemoveContainer" containerID="c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27"
Jul 22 13:02:32.304831 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:02:32.304813    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27\": container with ID starting with c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27 not found: ID does not exist" containerID="c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27"
Jul 22 13:02:32.304877 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.304838    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27"} err="failed to get container status \"c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27\": rpc error: code = NotFound desc = could not find container \"c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27\": container with ID starting with c68378538087c575929b9bd532054295ca9d1ba4805752ac3c3bb55d967c1b27 not found: ID does not exist"
Jul 22 13:02:32.324540 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.324498    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb" podStartSLOduration=1.3244845299999999 podStartE2EDuration="1.32448453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:02:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:02:32.323183003 +0000 UTC m=+504.615164037" watchObservedRunningTime="2026-07-22 13:02:32.32448453 +0000 UTC m=+504.616465561"
Jul 22 13:02:32.326448 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:02:32.326419    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fs4bp\" is forbidden: User \"system:node:ip-10-0-140-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-133.ec2.internal' and this object" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fs4bp"
Jul 22 13:02:32.947536 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:32.947503    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7754211f-d5d5-4ba4-b63d-c8f726b4b712" path="/var/lib/kubelet/pods/7754211f-d5d5-4ba4-b63d-c8f726b4b712/volumes"
Jul 22 13:02:43.304856 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:43.304826    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:48.173795 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.173749    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"]
Jul 22 13:02:48.174254 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.173979    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb" podUID="e554cc17-dd34-4183-9605-b87c75c2bff1" containerName="manager" containerID="cri-o://36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe" gracePeriod=10
Jul 22 13:02:48.512533 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.512453    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"]
Jul 22 13:02:48.516853 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.516830    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:02:48.540815 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.540786    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"]
Jul 22 13:02:48.642732 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.642703    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba821ce1-fbc7-475d-92e6-5668e920a4d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zwqbq\" (UID: \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:02:48.642889 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.642827    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5698s\" (UniqueName: \"kubernetes.io/projected/ba821ce1-fbc7-475d-92e6-5668e920a4d2-kube-api-access-5698s\") pod \"kuadrant-operator-controller-manager-86878c769f-zwqbq\" (UID: \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:02:48.727775 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.727749    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:48.744168 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.744135    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5698s\" (UniqueName: \"kubernetes.io/projected/ba821ce1-fbc7-475d-92e6-5668e920a4d2-kube-api-access-5698s\") pod \"kuadrant-operator-controller-manager-86878c769f-zwqbq\" (UID: \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:02:48.744336 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.744189    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba821ce1-fbc7-475d-92e6-5668e920a4d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zwqbq\" (UID: \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:02:48.744575 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.744554    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba821ce1-fbc7-475d-92e6-5668e920a4d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zwqbq\" (UID: \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:02:48.759730 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.759707    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5698s\" (UniqueName: \"kubernetes.io/projected/ba821ce1-fbc7-475d-92e6-5668e920a4d2-kube-api-access-5698s\") pod \"kuadrant-operator-controller-manager-86878c769f-zwqbq\" (UID: \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:02:48.827259 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.827231    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:02:48.844871 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.844843    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e554cc17-dd34-4183-9605-b87c75c2bff1-extensions-socket-volume\") pod \"e554cc17-dd34-4183-9605-b87c75c2bff1\" (UID: \"e554cc17-dd34-4183-9605-b87c75c2bff1\") "
Jul 22 13:02:48.845000 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.844971    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gh7gl\" (UniqueName: \"kubernetes.io/projected/e554cc17-dd34-4183-9605-b87c75c2bff1-kube-api-access-gh7gl\") pod \"e554cc17-dd34-4183-9605-b87c75c2bff1\" (UID: \"e554cc17-dd34-4183-9605-b87c75c2bff1\") "
Jul 22 13:02:48.845220 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.845199    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e554cc17-dd34-4183-9605-b87c75c2bff1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e554cc17-dd34-4183-9605-b87c75c2bff1" (UID: "e554cc17-dd34-4183-9605-b87c75c2bff1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:48.847302 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.847276    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e554cc17-dd34-4183-9605-b87c75c2bff1-kube-api-access-gh7gl" (OuterVolumeSpecName: "kube-api-access-gh7gl") pod "e554cc17-dd34-4183-9605-b87c75c2bff1" (UID: "e554cc17-dd34-4183-9605-b87c75c2bff1"). InnerVolumeSpecName "kube-api-access-gh7gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:02:48.945891 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.945863    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh7gl\" (UniqueName: \"kubernetes.io/projected/e554cc17-dd34-4183-9605-b87c75c2bff1-kube-api-access-gh7gl\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:02:48.945891 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.945887    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e554cc17-dd34-4183-9605-b87c75c2bff1-extensions-socket-volume\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:02:48.971972 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:48.971941    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"]
Jul 22 13:02:48.974964 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:02:48.974920    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba821ce1_fbc7_475d_92e6_5668e920a4d2.slice/crio-b43843b4f74fb7fbfc26f4c49fe9d77267c74233a687c53447cf4d377e1453c7 WatchSource:0}: Error finding container b43843b4f74fb7fbfc26f4c49fe9d77267c74233a687c53447cf4d377e1453c7: Status 404 returned error can't find the container with id b43843b4f74fb7fbfc26f4c49fe9d77267c74233a687c53447cf4d377e1453c7
Jul 22 13:02:49.372752 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.372721    2575 generic.go:356] "Generic (PLEG): container finished" podID="e554cc17-dd34-4183-9605-b87c75c2bff1" containerID="36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe" exitCode=0
Jul 22 13:02:49.373312 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.372762    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"
Jul 22 13:02:49.373312 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.372789    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb" event={"ID":"e554cc17-dd34-4183-9605-b87c75c2bff1","Type":"ContainerDied","Data":"36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe"}
Jul 22 13:02:49.373312 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.372840    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb" event={"ID":"e554cc17-dd34-4183-9605-b87c75c2bff1","Type":"ContainerDied","Data":"06b6a7d639545d7ec5b1108ed1b552885bbd18c2ae17116ed36b1fde90c4daad"}
Jul 22 13:02:49.373312 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.372865    2575 scope.go:117] "RemoveContainer" containerID="36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe"
Jul 22 13:02:49.374609 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.374572    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq" event={"ID":"ba821ce1-fbc7-475d-92e6-5668e920a4d2","Type":"ContainerStarted","Data":"3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7"}
Jul 22 13:02:49.374712 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.374625    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq" event={"ID":"ba821ce1-fbc7-475d-92e6-5668e920a4d2","Type":"ContainerStarted","Data":"b43843b4f74fb7fbfc26f4c49fe9d77267c74233a687c53447cf4d377e1453c7"}
Jul 22 13:02:49.374776 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.374728    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:02:49.381267 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.381250    2575 scope.go:117] "RemoveContainer" containerID="36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe"
Jul 22 13:02:49.381535 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:02:49.381519    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe\": container with ID starting with 36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe not found: ID does not exist" containerID="36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe"
Jul 22 13:02:49.381575 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.381545    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe"} err="failed to get container status \"36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe\": rpc error: code = NotFound desc = could not find container \"36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe\": container with ID starting with 36ed6cb9032e90b1809f9b5af14103b495b8f4282920954bbba9a4d9378585fe not found: ID does not exist"
Jul 22 13:02:49.398699 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.398668    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"]
Jul 22 13:02:49.404646 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.404619    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dqcvb"]
Jul 22 13:02:49.426302 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:49.426224    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq" podStartSLOduration=1.426204378 podStartE2EDuration="1.426204378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:02:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:02:49.426068357 +0000 UTC m=+521.718049414" watchObservedRunningTime="2026-07-22 13:02:49.426204378 +0000 UTC m=+521.718185408"
Jul 22 13:02:50.948045 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:02:50.948015    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e554cc17-dd34-4183-9605-b87c75c2bff1" path="/var/lib/kubelet/pods/e554cc17-dd34-4183-9605-b87c75c2bff1/volumes"
Jul 22 13:03:00.382132 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:00.382048    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:03:08.156229 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.156196    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-rhqpq"]
Jul 22 13:03:08.156632 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.156556    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e554cc17-dd34-4183-9605-b87c75c2bff1" containerName="manager"
Jul 22 13:03:08.156632 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.156567    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e554cc17-dd34-4183-9605-b87c75c2bff1" containerName="manager"
Jul 22 13:03:08.156703 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.156634    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e554cc17-dd34-4183-9605-b87c75c2bff1" containerName="manager"
Jul 22 13:03:08.158582 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.158567    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rhqpq"
Jul 22 13:03:08.162428 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.162404    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jvwpf\""
Jul 22 13:03:08.175135 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.175110    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rhqpq"]
Jul 22 13:03:08.208206 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.208173    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5sbg\" (UniqueName: \"kubernetes.io/projected/f932db76-5f4d-4aec-bc24-2e3d585100de-kube-api-access-p5sbg\") pod \"authorino-58cd955675-rhqpq\" (UID: \"f932db76-5f4d-4aec-bc24-2e3d585100de\") " pod="kuadrant-system/authorino-58cd955675-rhqpq"
Jul 22 13:03:08.309677 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.309572    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5sbg\" (UniqueName: \"kubernetes.io/projected/f932db76-5f4d-4aec-bc24-2e3d585100de-kube-api-access-p5sbg\") pod \"authorino-58cd955675-rhqpq\" (UID: \"f932db76-5f4d-4aec-bc24-2e3d585100de\") " pod="kuadrant-system/authorino-58cd955675-rhqpq"
Jul 22 13:03:08.321967 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.321940    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5sbg\" (UniqueName: \"kubernetes.io/projected/f932db76-5f4d-4aec-bc24-2e3d585100de-kube-api-access-p5sbg\") pod \"authorino-58cd955675-rhqpq\" (UID: \"f932db76-5f4d-4aec-bc24-2e3d585100de\") " pod="kuadrant-system/authorino-58cd955675-rhqpq"
Jul 22 13:03:08.467904 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.467822    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rhqpq"
Jul 22 13:03:08.590081 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.590052    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-tbr68"]
Jul 22 13:03:08.593036 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.593015    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tbr68"
Jul 22 13:03:08.602618 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.602594    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rhqpq"]
Jul 22 13:03:08.604664 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.604640    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tbr68"]
Jul 22 13:03:08.605034 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:03:08.605009    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf932db76_5f4d_4aec_bc24_2e3d585100de.slice/crio-1068e8a7cd2976d0d700e7b3053c21d8ff12bc8088b1319ee671cca9147334e8 WatchSource:0}: Error finding container 1068e8a7cd2976d0d700e7b3053c21d8ff12bc8088b1319ee671cca9147334e8: Status 404 returned error can't find the container with id 1068e8a7cd2976d0d700e7b3053c21d8ff12bc8088b1319ee671cca9147334e8
Jul 22 13:03:08.612967 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.612942    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq5lk\" (UniqueName: \"kubernetes.io/projected/65b151e8-f6d7-401c-b6c8-1c740d6ee060-kube-api-access-zq5lk\") pod \"authorino-75d556bd77-tbr68\" (UID: \"65b151e8-f6d7-401c-b6c8-1c740d6ee060\") " pod="kuadrant-system/authorino-75d556bd77-tbr68"
Jul 22 13:03:08.713885 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.713841    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq5lk\" (UniqueName: \"kubernetes.io/projected/65b151e8-f6d7-401c-b6c8-1c740d6ee060-kube-api-access-zq5lk\") pod \"authorino-75d556bd77-tbr68\" (UID: \"65b151e8-f6d7-401c-b6c8-1c740d6ee060\") " pod="kuadrant-system/authorino-75d556bd77-tbr68"
Jul 22 13:03:08.726108 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.726053    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq5lk\" (UniqueName: \"kubernetes.io/projected/65b151e8-f6d7-401c-b6c8-1c740d6ee060-kube-api-access-zq5lk\") pod \"authorino-75d556bd77-tbr68\" (UID: \"65b151e8-f6d7-401c-b6c8-1c740d6ee060\") " pod="kuadrant-system/authorino-75d556bd77-tbr68"
Jul 22 13:03:08.903678 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:08.903644    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tbr68"
Jul 22 13:03:09.035714 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:09.035689    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tbr68"]
Jul 22 13:03:09.038295 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:03:09.038265    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65b151e8_f6d7_401c_b6c8_1c740d6ee060.slice/crio-977a35e7a0d0625eadd7f6965bbb15f33d77c98308185c59bfce54ab1e9bddf5 WatchSource:0}: Error finding container 977a35e7a0d0625eadd7f6965bbb15f33d77c98308185c59bfce54ab1e9bddf5: Status 404 returned error can't find the container with id 977a35e7a0d0625eadd7f6965bbb15f33d77c98308185c59bfce54ab1e9bddf5
Jul 22 13:03:09.467318 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:09.467281    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tbr68" event={"ID":"65b151e8-f6d7-401c-b6c8-1c740d6ee060","Type":"ContainerStarted","Data":"977a35e7a0d0625eadd7f6965bbb15f33d77c98308185c59bfce54ab1e9bddf5"}
Jul 22 13:03:09.469629 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:09.469569    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rhqpq" event={"ID":"f932db76-5f4d-4aec-bc24-2e3d585100de","Type":"ContainerStarted","Data":"1068e8a7cd2976d0d700e7b3053c21d8ff12bc8088b1319ee671cca9147334e8"}
Jul 22 13:03:12.489388 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:12.489309    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tbr68" event={"ID":"65b151e8-f6d7-401c-b6c8-1c740d6ee060","Type":"ContainerStarted","Data":"d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc"}
Jul 22 13:03:12.490839 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:12.490815    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rhqpq" event={"ID":"f932db76-5f4d-4aec-bc24-2e3d585100de","Type":"ContainerStarted","Data":"01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837"}
Jul 22 13:03:12.515495 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:12.515448    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-tbr68" podStartSLOduration=1.49147075 podStartE2EDuration="4.515434792s" totalImagesPullingTime="3.023964042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:03:12.513966974 +0000 UTC m=+544.805948008" watchObservedRunningTime="2026-07-22 13:03:12.515434792 +0000 UTC m=+544.807415804"
Jul 22 13:03:12.540747 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:12.540685    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-rhqpq" podStartSLOduration=1.088597825 podStartE2EDuration="4.540662403s" totalImagesPullingTime="3.452064578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:03:12.53856221 +0000 UTC m=+544.830543253" watchObservedRunningTime="2026-07-22 13:03:12.540662403 +0000 UTC m=+544.832643437"
Jul 22 13:03:12.589532 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:12.589500    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rhqpq"]
Jul 22 13:03:14.498661 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:14.498621    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-rhqpq" podUID="f932db76-5f4d-4aec-bc24-2e3d585100de" containerName="authorino" containerID="cri-o://01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837" gracePeriod=30
Jul 22 13:03:14.744157 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:14.744137    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rhqpq"
Jul 22 13:03:14.870216 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:14.870190    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5sbg\" (UniqueName: \"kubernetes.io/projected/f932db76-5f4d-4aec-bc24-2e3d585100de-kube-api-access-p5sbg\") pod \"f932db76-5f4d-4aec-bc24-2e3d585100de\" (UID: \"f932db76-5f4d-4aec-bc24-2e3d585100de\") "
Jul 22 13:03:14.872229 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:14.872203    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f932db76-5f4d-4aec-bc24-2e3d585100de-kube-api-access-p5sbg" (OuterVolumeSpecName: "kube-api-access-p5sbg") pod "f932db76-5f4d-4aec-bc24-2e3d585100de" (UID: "f932db76-5f4d-4aec-bc24-2e3d585100de"). InnerVolumeSpecName "kube-api-access-p5sbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:03:14.970924 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:14.970875    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5sbg\" (UniqueName: \"kubernetes.io/projected/f932db76-5f4d-4aec-bc24-2e3d585100de-kube-api-access-p5sbg\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:03:15.503833 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:15.503802    2575 generic.go:356] "Generic (PLEG): container finished" podID="f932db76-5f4d-4aec-bc24-2e3d585100de" containerID="01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837" exitCode=0
Jul 22 13:03:15.504244 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:15.503846    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rhqpq"
Jul 22 13:03:15.504244 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:15.503887    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rhqpq" event={"ID":"f932db76-5f4d-4aec-bc24-2e3d585100de","Type":"ContainerDied","Data":"01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837"}
Jul 22 13:03:15.504244 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:15.503938    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rhqpq" event={"ID":"f932db76-5f4d-4aec-bc24-2e3d585100de","Type":"ContainerDied","Data":"1068e8a7cd2976d0d700e7b3053c21d8ff12bc8088b1319ee671cca9147334e8"}
Jul 22 13:03:15.504244 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:15.503957    2575 scope.go:117] "RemoveContainer" containerID="01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837"
Jul 22 13:03:15.512247 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:15.512230    2575 scope.go:117] "RemoveContainer" containerID="01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837"
Jul 22 13:03:15.512517 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:03:15.512499    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837\": container with ID starting with 01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837 not found: ID does not exist" containerID="01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837"
Jul 22 13:03:15.512566 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:15.512526    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837"} err="failed to get container status \"01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837\": rpc error: code = NotFound desc = could not find container \"01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837\": container with ID starting with 01e9fe402366cda0dbdcf04d190f1980a5ce536656425acf59ffc723962f8837 not found: ID does not exist"
Jul 22 13:03:15.523783 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:15.523761    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rhqpq"]
Jul 22 13:03:15.528390 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:15.528368    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-rhqpq"]
Jul 22 13:03:16.947453 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:16.947421    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f932db76-5f4d-4aec-bc24-2e3d585100de" path="/var/lib/kubelet/pods/f932db76-5f4d-4aec-bc24-2e3d585100de/volumes"
Jul 22 13:03:41.025900 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.025869    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mbkmn"]
Jul 22 13:03:41.026516 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.026404    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f932db76-5f4d-4aec-bc24-2e3d585100de" containerName="authorino"
Jul 22 13:03:41.026516 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.026425    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f932db76-5f4d-4aec-bc24-2e3d585100de" containerName="authorino"
Jul 22 13:03:41.026633 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.026517    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f932db76-5f4d-4aec-bc24-2e3d585100de" containerName="authorino"
Jul 22 13:03:41.029771 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.029752    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mbkmn"
Jul 22 13:03:41.037542 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.037248    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mbkmn"]
Jul 22 13:03:41.194425 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.194390    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmkzh\" (UniqueName: \"kubernetes.io/projected/eaf85e25-8996-45ae-8131-fd0a3f4a581b-kube-api-access-jmkzh\") pod \"authorino-5d4b67c446-mbkmn\" (UID: \"eaf85e25-8996-45ae-8131-fd0a3f4a581b\") " pod="kuadrant-system/authorino-5d4b67c446-mbkmn"
Jul 22 13:03:41.281471 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.281378    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mbkmn"]
Jul 22 13:03:41.281656 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:03:41.281636    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jmkzh], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-mbkmn" podUID="eaf85e25-8996-45ae-8131-fd0a3f4a581b"
Jul 22 13:03:41.295082 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.295048    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmkzh\" (UniqueName: \"kubernetes.io/projected/eaf85e25-8996-45ae-8131-fd0a3f4a581b-kube-api-access-jmkzh\") pod \"authorino-5d4b67c446-mbkmn\" (UID: \"eaf85e25-8996-45ae-8131-fd0a3f4a581b\") " pod="kuadrant-system/authorino-5d4b67c446-mbkmn"
Jul 22 13:03:41.307672 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.307647    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmkzh\" (UniqueName: \"kubernetes.io/projected/eaf85e25-8996-45ae-8131-fd0a3f4a581b-kube-api-access-jmkzh\") pod \"authorino-5d4b67c446-mbkmn\" (UID: \"eaf85e25-8996-45ae-8131-fd0a3f4a581b\") " pod="kuadrant-system/authorino-5d4b67c446-mbkmn"
Jul 22 13:03:41.314872 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.314841    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68c4c669cb-f5vzv"]
Jul 22 13:03:41.317186 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.317167    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68c4c669cb-f5vzv"
Jul 22 13:03:41.325040 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.325014    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68c4c669cb-f5vzv"]
Jul 22 13:03:41.388426 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.388384    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68c4c669cb-f5vzv"]
Jul 22 13:03:41.388704 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:03:41.388681    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mrs45], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-68c4c669cb-f5vzv" podUID="1df27e3e-9883-402e-b97d-c369e2d6c478"
Jul 22 13:03:41.396128 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.396101    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrs45\" (UniqueName: \"kubernetes.io/projected/1df27e3e-9883-402e-b97d-c369e2d6c478-kube-api-access-mrs45\") pod \"authorino-68c4c669cb-f5vzv\" (UID: \"1df27e3e-9883-402e-b97d-c369e2d6c478\") " pod="kuadrant-system/authorino-68c4c669cb-f5vzv"
Jul 22 13:03:41.419757 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.419718    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85db8bd7cb-hm5xf"]
Jul 22 13:03:41.422144 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.422123    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:03:41.424684 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.424663    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 22 13:03:41.430570 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.430548    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85db8bd7cb-hm5xf"]
Jul 22 13:03:41.497306 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.497271    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrs45\" (UniqueName: \"kubernetes.io/projected/1df27e3e-9883-402e-b97d-c369e2d6c478-kube-api-access-mrs45\") pod \"authorino-68c4c669cb-f5vzv\" (UID: \"1df27e3e-9883-402e-b97d-c369e2d6c478\") " pod="kuadrant-system/authorino-68c4c669cb-f5vzv"
Jul 22 13:03:41.497487 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.497331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-tls-cert\") pod \"authorino-85db8bd7cb-hm5xf\" (UID: \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\") " pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:03:41.497487 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.497349    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hts7\" (UniqueName: \"kubernetes.io/projected/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-kube-api-access-8hts7\") pod \"authorino-85db8bd7cb-hm5xf\" (UID: \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\") " pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:03:41.507583 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.507556    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrs45\" (UniqueName: \"kubernetes.io/projected/1df27e3e-9883-402e-b97d-c369e2d6c478-kube-api-access-mrs45\") pod \"authorino-68c4c669cb-f5vzv\" (UID: \"1df27e3e-9883-402e-b97d-c369e2d6c478\") " pod="kuadrant-system/authorino-68c4c669cb-f5vzv"
Jul 22 13:03:41.598827 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.598789    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-tls-cert\") pod \"authorino-85db8bd7cb-hm5xf\" (UID: \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\") " pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:03:41.598827 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.598826    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hts7\" (UniqueName: \"kubernetes.io/projected/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-kube-api-access-8hts7\") pod \"authorino-85db8bd7cb-hm5xf\" (UID: \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\") " pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:03:41.601378 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.601357    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-tls-cert\") pod \"authorino-85db8bd7cb-hm5xf\" (UID: \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\") " pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:03:41.612097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.612069    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hts7\" (UniqueName: \"kubernetes.io/projected/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-kube-api-access-8hts7\") pod \"authorino-85db8bd7cb-hm5xf\" (UID: \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\") " pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:03:41.622595 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.622574    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mbkmn"
Jul 22 13:03:41.622739 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.622573    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68c4c669cb-f5vzv"
Jul 22 13:03:41.628882 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.628862    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mbkmn"
Jul 22 13:03:41.631990 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.631971    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68c4c669cb-f5vzv"
Jul 22 13:03:41.731880 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.731850    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:03:41.800503 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.800478    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmkzh\" (UniqueName: \"kubernetes.io/projected/eaf85e25-8996-45ae-8131-fd0a3f4a581b-kube-api-access-jmkzh\") pod \"eaf85e25-8996-45ae-8131-fd0a3f4a581b\" (UID: \"eaf85e25-8996-45ae-8131-fd0a3f4a581b\") "
Jul 22 13:03:41.800642 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.800586    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrs45\" (UniqueName: \"kubernetes.io/projected/1df27e3e-9883-402e-b97d-c369e2d6c478-kube-api-access-mrs45\") pod \"1df27e3e-9883-402e-b97d-c369e2d6c478\" (UID: \"1df27e3e-9883-402e-b97d-c369e2d6c478\") "
Jul 22 13:03:41.802565 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.802534    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf85e25-8996-45ae-8131-fd0a3f4a581b-kube-api-access-jmkzh" (OuterVolumeSpecName: "kube-api-access-jmkzh") pod "eaf85e25-8996-45ae-8131-fd0a3f4a581b" (UID: "eaf85e25-8996-45ae-8131-fd0a3f4a581b"). InnerVolumeSpecName "kube-api-access-jmkzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:03:41.802735 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.802716    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1df27e3e-9883-402e-b97d-c369e2d6c478-kube-api-access-mrs45" (OuterVolumeSpecName: "kube-api-access-mrs45") pod "1df27e3e-9883-402e-b97d-c369e2d6c478" (UID: "1df27e3e-9883-402e-b97d-c369e2d6c478"). InnerVolumeSpecName "kube-api-access-mrs45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:03:41.866258 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.866233    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85db8bd7cb-hm5xf"]
Jul 22 13:03:41.868106 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:03:41.868081    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20a04d8b_b01d_48bb_bea2_6ac57a32d54e.slice/crio-6d0aa8bab1d62e2a69980225dc4a104686ddd01ed8e63d7f065cc77f65728ce9 WatchSource:0}: Error finding container 6d0aa8bab1d62e2a69980225dc4a104686ddd01ed8e63d7f065cc77f65728ce9: Status 404 returned error can't find the container with id 6d0aa8bab1d62e2a69980225dc4a104686ddd01ed8e63d7f065cc77f65728ce9
Jul 22 13:03:41.901757 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.901730    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrs45\" (UniqueName: \"kubernetes.io/projected/1df27e3e-9883-402e-b97d-c369e2d6c478-kube-api-access-mrs45\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:03:41.901757 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:41.901755    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmkzh\" (UniqueName: \"kubernetes.io/projected/eaf85e25-8996-45ae-8131-fd0a3f4a581b-kube-api-access-jmkzh\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:03:42.633233 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.633199    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68c4c669cb-f5vzv"
Jul 22 13:03:42.633233 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.633223    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf" event={"ID":"20a04d8b-b01d-48bb-bea2-6ac57a32d54e","Type":"ContainerStarted","Data":"183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d"}
Jul 22 13:03:42.633787 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.633259    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf" event={"ID":"20a04d8b-b01d-48bb-bea2-6ac57a32d54e","Type":"ContainerStarted","Data":"6d0aa8bab1d62e2a69980225dc4a104686ddd01ed8e63d7f065cc77f65728ce9"}
Jul 22 13:03:42.633787 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.633372    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mbkmn"
Jul 22 13:03:42.654000 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.653951    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf" podStartSLOduration=1.1465890189999999 podStartE2EDuration="1.65391055s" totalImagesPullingTime="507.321531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:03:42.653139693 +0000 UTC m=+574.945120753" watchObservedRunningTime="2026-07-22 13:03:42.65391055 +0000 UTC m=+574.945891628"
Jul 22 13:03:42.678970 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.678919    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mbkmn"]
Jul 22 13:03:42.683411 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.683379    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tbr68"]
Jul 22 13:03:42.683613 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.683590    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-tbr68" podUID="65b151e8-f6d7-401c-b6c8-1c740d6ee060" containerName="authorino" containerID="cri-o://d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc" gracePeriod=30
Jul 22 13:03:42.686482 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.686447    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mbkmn"]
Jul 22 13:03:42.715116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.715086    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68c4c669cb-f5vzv"]
Jul 22 13:03:42.722143 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.722113    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-68c4c669cb-f5vzv"]
Jul 22 13:03:42.930389 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.930366    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tbr68"
Jul 22 13:03:42.948723 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.948688    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1df27e3e-9883-402e-b97d-c369e2d6c478" path="/var/lib/kubelet/pods/1df27e3e-9883-402e-b97d-c369e2d6c478/volumes"
Jul 22 13:03:42.949021 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:42.949005    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaf85e25-8996-45ae-8131-fd0a3f4a581b" path="/var/lib/kubelet/pods/eaf85e25-8996-45ae-8131-fd0a3f4a581b/volumes"
Jul 22 13:03:43.013729 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.013694    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq5lk\" (UniqueName: \"kubernetes.io/projected/65b151e8-f6d7-401c-b6c8-1c740d6ee060-kube-api-access-zq5lk\") pod \"65b151e8-f6d7-401c-b6c8-1c740d6ee060\" (UID: \"65b151e8-f6d7-401c-b6c8-1c740d6ee060\") "
Jul 22 13:03:43.015728 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.015697    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b151e8-f6d7-401c-b6c8-1c740d6ee060-kube-api-access-zq5lk" (OuterVolumeSpecName: "kube-api-access-zq5lk") pod "65b151e8-f6d7-401c-b6c8-1c740d6ee060" (UID: "65b151e8-f6d7-401c-b6c8-1c740d6ee060"). InnerVolumeSpecName "kube-api-access-zq5lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:03:43.114545 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.114504    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq5lk\" (UniqueName: \"kubernetes.io/projected/65b151e8-f6d7-401c-b6c8-1c740d6ee060-kube-api-access-zq5lk\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:03:43.638917 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.638880    2575 generic.go:356] "Generic (PLEG): container finished" podID="65b151e8-f6d7-401c-b6c8-1c740d6ee060" containerID="d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc" exitCode=0
Jul 22 13:03:43.638917 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.638915    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tbr68"
Jul 22 13:03:43.639383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.638959    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tbr68" event={"ID":"65b151e8-f6d7-401c-b6c8-1c740d6ee060","Type":"ContainerDied","Data":"d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc"}
Jul 22 13:03:43.639383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.638997    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tbr68" event={"ID":"65b151e8-f6d7-401c-b6c8-1c740d6ee060","Type":"ContainerDied","Data":"977a35e7a0d0625eadd7f6965bbb15f33d77c98308185c59bfce54ab1e9bddf5"}
Jul 22 13:03:43.639383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.639015    2575 scope.go:117] "RemoveContainer" containerID="d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc"
Jul 22 13:03:43.647692 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.647674    2575 scope.go:117] "RemoveContainer" containerID="d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc"
Jul 22 13:03:43.647982 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:03:43.647959    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc\": container with ID starting with d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc not found: ID does not exist" containerID="d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc"
Jul 22 13:03:43.648093 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.647992    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc"} err="failed to get container status \"d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc\": rpc error: code = NotFound desc = could not find container \"d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc\": container with ID starting with d9df3efdfa4a0b9069a535f19522afb18dbdc0aad270202e89635679704171fc not found: ID does not exist"
Jul 22 13:03:43.663025 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.663000    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tbr68"]
Jul 22 13:03:43.670781 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:43.670756    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tbr68"]
Jul 22 13:03:44.948067 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:03:44.948036    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65b151e8-f6d7-401c-b6c8-1c740d6ee060" path="/var/lib/kubelet/pods/65b151e8-f6d7-401c-b6c8-1c740d6ee060/volumes"
Jul 22 13:04:08.818694 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:08.818651    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:04:08.820008 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:08.819986    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:04:56.348497 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.348464    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"]
Jul 22 13:04:56.348991 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.348796    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b151e8-f6d7-401c-b6c8-1c740d6ee060" containerName="authorino"
Jul 22 13:04:56.348991 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.348807    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b151e8-f6d7-401c-b6c8-1c740d6ee060" containerName="authorino"
Jul 22 13:04:56.348991 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.348867    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b151e8-f6d7-401c-b6c8-1c740d6ee060" containerName="authorino"
Jul 22 13:04:56.351790 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.351765    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.355743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.355717    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 13:04:56.355743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.355723    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vvhw8\""
Jul 22 13:04:56.355743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.355752    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 22 13:04:56.356047 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.355819    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 13:04:56.370060 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.370035    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"]
Jul 22 13:04:56.457144 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.457110    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.457337 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.457157    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.457337 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.457174    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.457337 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.457197    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qchx\" (UniqueName: \"kubernetes.io/projected/c1936ed5-dd0f-47a1-872b-5b624998506d-kube-api-access-4qchx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.457337 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.457240    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.457337 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.457319    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1936ed5-dd0f-47a1-872b-5b624998506d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.558681 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.558646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.558860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.558721    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1936ed5-dd0f-47a1-872b-5b624998506d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.558860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.558751    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.558860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.558801    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.558860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.558825    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.558860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.558850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qchx\" (UniqueName: \"kubernetes.io/projected/c1936ed5-dd0f-47a1-872b-5b624998506d-kube-api-access-4qchx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.559222 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.559200    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.559272 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.559225    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.559312 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.559273    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.561063 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.561044    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1936ed5-dd0f-47a1-872b-5b624998506d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.561207 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.561191    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1936ed5-dd0f-47a1-872b-5b624998506d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.569273 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.569249    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qchx\" (UniqueName: \"kubernetes.io/projected/c1936ed5-dd0f-47a1-872b-5b624998506d-kube-api-access-4qchx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r8c59\" (UID: \"c1936ed5-dd0f-47a1-872b-5b624998506d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.662171 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.662079    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:04:56.799691 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.799655    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"]
Jul 22 13:04:56.801979 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:04:56.801945    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1936ed5_dd0f_47a1_872b_5b624998506d.slice/crio-80b793b2edc8aa41aa82eb48b00fae88d89b33752120fc5800e619ba74111ff2 WatchSource:0}: Error finding container 80b793b2edc8aa41aa82eb48b00fae88d89b33752120fc5800e619ba74111ff2: Status 404 returned error can't find the container with id 80b793b2edc8aa41aa82eb48b00fae88d89b33752120fc5800e619ba74111ff2
Jul 22 13:04:56.963150 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:04:56.963062    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59" event={"ID":"c1936ed5-dd0f-47a1-872b-5b624998506d","Type":"ContainerStarted","Data":"80b793b2edc8aa41aa82eb48b00fae88d89b33752120fc5800e619ba74111ff2"}
Jul 22 13:05:00.982959 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:00.982905    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59" event={"ID":"c1936ed5-dd0f-47a1-872b-5b624998506d","Type":"ContainerStarted","Data":"61d61c1bf052659bbccf560625265462083e37c4f9034b5ed258d5ab67beeacb"}
Jul 22 13:05:01.013655 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:01.013599    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59" podStartSLOduration=1.491549861 podStartE2EDuration="5.01358273s" totalImagesPullingTime="3.522032869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:05:01.012156268 +0000 UTC m=+653.304137304" watchObservedRunningTime="2026-07-22 13:05:01.01358273 +0000 UTC m=+653.305563762"
Jul 22 13:05:06.662773 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:06.662728    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:05:06.663196 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:06.662803    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:05:06.675036 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:06.675008    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:05:07.021064 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:07.020985    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r8c59"
Jul 22 13:05:10.036345 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.036312    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"]
Jul 22 13:05:10.039677 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.039661    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.043134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.043114    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 22 13:05:10.049991 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.049955    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"]
Jul 22 13:05:10.181605 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.181565    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.181799 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.181621    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.181799 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.181639    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.181799 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.181662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63839fd0-46f5-4abb-b225-1ffb52e19f5d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.181799 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.181707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h66km\" (UniqueName: \"kubernetes.io/projected/63839fd0-46f5-4abb-b225-1ffb52e19f5d-kube-api-access-h66km\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.181799 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.181733    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.283225 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.283191    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.283225 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.283235    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.283456 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.283251    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.283456 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.283272    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63839fd0-46f5-4abb-b225-1ffb52e19f5d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.283456 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.283325    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h66km\" (UniqueName: \"kubernetes.io/projected/63839fd0-46f5-4abb-b225-1ffb52e19f5d-kube-api-access-h66km\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.283456 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.283356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.283664 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.283642    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.283749 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.283716    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.283786 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.283760    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.285584 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.285564    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63839fd0-46f5-4abb-b225-1ffb52e19f5d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.285819 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.285800    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63839fd0-46f5-4abb-b225-1ffb52e19f5d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.294069 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.294007    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h66km\" (UniqueName: \"kubernetes.io/projected/63839fd0-46f5-4abb-b225-1ffb52e19f5d-kube-api-access-h66km\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xcth2\" (UID: \"63839fd0-46f5-4abb-b225-1ffb52e19f5d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.349882 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.349843    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:10.505790 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:05:10.505758    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63839fd0_46f5_4abb_b225_1ffb52e19f5d.slice/crio-980d4af0915ebbd17b9802f1d09a96f04c3e32ca0c88d3ad80b046c6a67cc173 WatchSource:0}: Error finding container 980d4af0915ebbd17b9802f1d09a96f04c3e32ca0c88d3ad80b046c6a67cc173: Status 404 returned error can't find the container with id 980d4af0915ebbd17b9802f1d09a96f04c3e32ca0c88d3ad80b046c6a67cc173
Jul 22 13:05:10.506702 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.506683    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"]
Jul 22 13:05:10.507628 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:10.507609    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 13:05:11.034954 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:11.034896    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2" event={"ID":"63839fd0-46f5-4abb-b225-1ffb52e19f5d","Type":"ContainerStarted","Data":"084dbbc68fdf487fc33d37778f5946e912f2154dd8de0e76b1fe9deeb74d5d8c"}
Jul 22 13:05:11.035292 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:11.035270    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2" event={"ID":"63839fd0-46f5-4abb-b225-1ffb52e19f5d","Type":"ContainerStarted","Data":"980d4af0915ebbd17b9802f1d09a96f04c3e32ca0c88d3ad80b046c6a67cc173"}
Jul 22 13:05:11.066901 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:11.066795    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2" podStartSLOduration=0.787560715 podStartE2EDuration="1.066779631s" totalImagesPullingTime="279.218916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:05:11.066063897 +0000 UTC m=+663.358044963" watchObservedRunningTime="2026-07-22 13:05:11.066779631 +0000 UTC m=+663.358760664"
Jul 22 13:05:20.350223 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:20.350187    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:20.350673 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:20.350384    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:20.362773 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:20.362750    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:21.099825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:21.099797    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xcth2"
Jul 22 13:05:53.554007 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.553972    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"]
Jul 22 13:05:53.559466 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.559444    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.562000 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.561980    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 22 13:05:53.587971 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.587946    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"]
Jul 22 13:05:53.671822 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.671786    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bgc6\" (UniqueName: \"kubernetes.io/projected/82890132-3dd0-4885-bae0-e5fb6dd7b308-kube-api-access-4bgc6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.671822 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.671828    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.672124 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.671951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.672124 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.671987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.672124 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.672051    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82890132-3dd0-4885-bae0-e5fb6dd7b308-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.672124 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.672112    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.773012 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.772977    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bgc6\" (UniqueName: \"kubernetes.io/projected/82890132-3dd0-4885-bae0-e5fb6dd7b308-kube-api-access-4bgc6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.773179 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.773025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.773179 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.773084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.773179 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.773111    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.773179 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.773155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82890132-3dd0-4885-bae0-e5fb6dd7b308-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.773369 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.773203    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.773556 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.773533    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.773556 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.773549    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.773674 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.773594    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.775441 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.775420    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82890132-3dd0-4885-bae0-e5fb6dd7b308-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.775742 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.775721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82890132-3dd0-4885-bae0-e5fb6dd7b308-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.783192 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.783169    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bgc6\" (UniqueName: \"kubernetes.io/projected/82890132-3dd0-4885-bae0-e5fb6dd7b308-kube-api-access-4bgc6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px5xp\" (UID: \"82890132-3dd0-4885-bae0-e5fb6dd7b308\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:53.871117 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:53.871085    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:05:54.000325 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:54.000298    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"]
Jul 22 13:05:54.002280 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:05:54.002254    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82890132_3dd0_4885_bae0_e5fb6dd7b308.slice/crio-cd26a545bffdbc10ae7e6356f298a967325c5acf5eff69367734fff90723e983 WatchSource:0}: Error finding container cd26a545bffdbc10ae7e6356f298a967325c5acf5eff69367734fff90723e983: Status 404 returned error can't find the container with id cd26a545bffdbc10ae7e6356f298a967325c5acf5eff69367734fff90723e983
Jul 22 13:05:54.230288 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:54.230196    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp" event={"ID":"82890132-3dd0-4885-bae0-e5fb6dd7b308","Type":"ContainerStarted","Data":"cd26a545bffdbc10ae7e6356f298a967325c5acf5eff69367734fff90723e983"}
Jul 22 13:05:55.236244 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:55.236205    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp" event={"ID":"82890132-3dd0-4885-bae0-e5fb6dd7b308","Type":"ContainerStarted","Data":"a3ef8ba51bd682190985f449ea83e93340f7c69b4124e7f5ae67549fce3fa208"}
Jul 22 13:05:55.256198 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:05:55.256141    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp" podStartSLOduration=1.9624537119999999 podStartE2EDuration="2.256124468s" totalImagesPullingTime="293.670756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:05:55.253949876 +0000 UTC m=+707.545930931" watchObservedRunningTime="2026-07-22 13:05:55.256124468 +0000 UTC m=+707.548105501"
Jul 22 13:06:03.872142 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:06:03.872035    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:06:03.872142 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:06:03.872110    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:06:03.885655 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:06:03.885618    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:06:04.292003 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:06:04.291910    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px5xp"
Jul 22 13:07:56.006803 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.006722    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bd7fb4977-n7lqs"]
Jul 22 13:07:56.009865 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.009848    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bd7fb4977-n7lqs"
Jul 22 13:07:56.021743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.021716    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bd7fb4977-n7lqs"]
Jul 22 13:07:56.074977 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.074943    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fhjq\" (UniqueName: \"kubernetes.io/projected/1d2ae148-3643-41cd-a1c1-00dea6fd3a0f-kube-api-access-9fhjq\") pod \"authorino-7bd7fb4977-n7lqs\" (UID: \"1d2ae148-3643-41cd-a1c1-00dea6fd3a0f\") " pod="kuadrant-system/authorino-7bd7fb4977-n7lqs"
Jul 22 13:07:56.075166 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.075026    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d2ae148-3643-41cd-a1c1-00dea6fd3a0f-tls-cert\") pod \"authorino-7bd7fb4977-n7lqs\" (UID: \"1d2ae148-3643-41cd-a1c1-00dea6fd3a0f\") " pod="kuadrant-system/authorino-7bd7fb4977-n7lqs"
Jul 22 13:07:56.175746 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.175710    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fhjq\" (UniqueName: \"kubernetes.io/projected/1d2ae148-3643-41cd-a1c1-00dea6fd3a0f-kube-api-access-9fhjq\") pod \"authorino-7bd7fb4977-n7lqs\" (UID: \"1d2ae148-3643-41cd-a1c1-00dea6fd3a0f\") " pod="kuadrant-system/authorino-7bd7fb4977-n7lqs"
Jul 22 13:07:56.175989 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.175785    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d2ae148-3643-41cd-a1c1-00dea6fd3a0f-tls-cert\") pod \"authorino-7bd7fb4977-n7lqs\" (UID: \"1d2ae148-3643-41cd-a1c1-00dea6fd3a0f\") " pod="kuadrant-system/authorino-7bd7fb4977-n7lqs"
Jul 22 13:07:56.178306 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.178287    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d2ae148-3643-41cd-a1c1-00dea6fd3a0f-tls-cert\") pod \"authorino-7bd7fb4977-n7lqs\" (UID: \"1d2ae148-3643-41cd-a1c1-00dea6fd3a0f\") " pod="kuadrant-system/authorino-7bd7fb4977-n7lqs"
Jul 22 13:07:56.185533 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.185501    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fhjq\" (UniqueName: \"kubernetes.io/projected/1d2ae148-3643-41cd-a1c1-00dea6fd3a0f-kube-api-access-9fhjq\") pod \"authorino-7bd7fb4977-n7lqs\" (UID: \"1d2ae148-3643-41cd-a1c1-00dea6fd3a0f\") " pod="kuadrant-system/authorino-7bd7fb4977-n7lqs"
Jul 22 13:07:56.318889 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.318795    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bd7fb4977-n7lqs"
Jul 22 13:07:56.464826 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.464799    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bd7fb4977-n7lqs"]
Jul 22 13:07:56.467586 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:07:56.467554    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d2ae148_3643_41cd_a1c1_00dea6fd3a0f.slice/crio-a00f9422673f6240dc955a42809cc35ebc8cdd94994ba16c99189f77433068e0 WatchSource:0}: Error finding container a00f9422673f6240dc955a42809cc35ebc8cdd94994ba16c99189f77433068e0: Status 404 returned error can't find the container with id a00f9422673f6240dc955a42809cc35ebc8cdd94994ba16c99189f77433068e0
Jul 22 13:07:56.778820 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:56.778786    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bd7fb4977-n7lqs" event={"ID":"1d2ae148-3643-41cd-a1c1-00dea6fd3a0f","Type":"ContainerStarted","Data":"a00f9422673f6240dc955a42809cc35ebc8cdd94994ba16c99189f77433068e0"}
Jul 22 13:07:57.786288 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:57.786250    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bd7fb4977-n7lqs" event={"ID":"1d2ae148-3643-41cd-a1c1-00dea6fd3a0f","Type":"ContainerStarted","Data":"d1b8460be0e17e7be54e2d2661b8dc63d6ba24ae1015158f7a2d72eb5d3a145e"}
Jul 22 13:07:57.839071 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:57.838989    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bd7fb4977-n7lqs" podStartSLOduration=2.3671481070000002 podStartE2EDuration="2.838965832s" totalImagesPullingTime="471.817725ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:07:57.837295611 +0000 UTC m=+830.129276645" watchObservedRunningTime="2026-07-22 13:07:57.838965832 +0000 UTC m=+830.130946867"
Jul 22 13:07:57.877138 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:57.875959    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85db8bd7cb-hm5xf"]
Jul 22 13:07:57.877138 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:57.876163    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf" podUID="20a04d8b-b01d-48bb-bea2-6ac57a32d54e" containerName="authorino" containerID="cri-o://183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d" gracePeriod=30
Jul 22 13:07:58.145640 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.145615    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:07:58.294895 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.294859    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hts7\" (UniqueName: \"kubernetes.io/projected/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-kube-api-access-8hts7\") pod \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\" (UID: \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\") "
Jul 22 13:07:58.295263 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.295242    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-tls-cert\") pod \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\" (UID: \"20a04d8b-b01d-48bb-bea2-6ac57a32d54e\") "
Jul 22 13:07:58.306143 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.306067    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-kube-api-access-8hts7" (OuterVolumeSpecName: "kube-api-access-8hts7") pod "20a04d8b-b01d-48bb-bea2-6ac57a32d54e" (UID: "20a04d8b-b01d-48bb-bea2-6ac57a32d54e"). InnerVolumeSpecName "kube-api-access-8hts7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:07:58.316838 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.316809    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "20a04d8b-b01d-48bb-bea2-6ac57a32d54e" (UID: "20a04d8b-b01d-48bb-bea2-6ac57a32d54e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:07:58.396888 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.396854    2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-tls-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:07:58.396888 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.396886    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hts7\" (UniqueName: \"kubernetes.io/projected/20a04d8b-b01d-48bb-bea2-6ac57a32d54e-kube-api-access-8hts7\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:07:58.791939 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.791897    2575 generic.go:356] "Generic (PLEG): container finished" podID="20a04d8b-b01d-48bb-bea2-6ac57a32d54e" containerID="183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d" exitCode=0
Jul 22 13:07:58.792361 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.791977    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf"
Jul 22 13:07:58.792361 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.791992    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf" event={"ID":"20a04d8b-b01d-48bb-bea2-6ac57a32d54e","Type":"ContainerDied","Data":"183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d"}
Jul 22 13:07:58.792361 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.792030    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85db8bd7cb-hm5xf" event={"ID":"20a04d8b-b01d-48bb-bea2-6ac57a32d54e","Type":"ContainerDied","Data":"6d0aa8bab1d62e2a69980225dc4a104686ddd01ed8e63d7f065cc77f65728ce9"}
Jul 22 13:07:58.792361 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.792047    2575 scope.go:117] "RemoveContainer" containerID="183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d"
Jul 22 13:07:58.800739 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.800722    2575 scope.go:117] "RemoveContainer" containerID="183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d"
Jul 22 13:07:58.801024 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:07:58.801001    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d\": container with ID starting with 183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d not found: ID does not exist" containerID="183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d"
Jul 22 13:07:58.801089 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.801032    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d"} err="failed to get container status \"183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d\": rpc error: code = NotFound desc = could not find container \"183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d\": container with ID starting with 183be7e5a0aa940f7ca1fe717effb5f9c0bde19d1764d6c50b3ca8e4e4c2749d not found: ID does not exist"
Jul 22 13:07:58.815818 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.815790    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85db8bd7cb-hm5xf"]
Jul 22 13:07:58.821052 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.821030    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-85db8bd7cb-hm5xf"]
Jul 22 13:07:58.948056 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:07:58.948024    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20a04d8b-b01d-48bb-bea2-6ac57a32d54e" path="/var/lib/kubelet/pods/20a04d8b-b01d-48bb-bea2-6ac57a32d54e/volumes"
Jul 22 13:09:08.850430 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:09:08.850401    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:09:08.850902 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:09:08.850401    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:14:08.878512 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:08.878484    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:14:08.879463 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:08.879438    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:14:54.140286 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:54.140250    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"]
Jul 22 13:14:54.140794 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:54.140466    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq" podUID="ba821ce1-fbc7-475d-92e6-5668e920a4d2" containerName="manager" containerID="cri-o://3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7" gracePeriod=10
Jul 22 13:14:57.399833 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.399811    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:14:57.491967 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.491857    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba821ce1-fbc7-475d-92e6-5668e920a4d2-extensions-socket-volume\") pod \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\" (UID: \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\") "
Jul 22 13:14:57.491967 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.491968    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5698s\" (UniqueName: \"kubernetes.io/projected/ba821ce1-fbc7-475d-92e6-5668e920a4d2-kube-api-access-5698s\") pod \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\" (UID: \"ba821ce1-fbc7-475d-92e6-5668e920a4d2\") "
Jul 22 13:14:57.492241 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.492220    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba821ce1-fbc7-475d-92e6-5668e920a4d2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ba821ce1-fbc7-475d-92e6-5668e920a4d2" (UID: "ba821ce1-fbc7-475d-92e6-5668e920a4d2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:14:57.493965 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.493946    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba821ce1-fbc7-475d-92e6-5668e920a4d2-kube-api-access-5698s" (OuterVolumeSpecName: "kube-api-access-5698s") pod "ba821ce1-fbc7-475d-92e6-5668e920a4d2" (UID: "ba821ce1-fbc7-475d-92e6-5668e920a4d2"). InnerVolumeSpecName "kube-api-access-5698s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:14:57.592839 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.592796    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5698s\" (UniqueName: \"kubernetes.io/projected/ba821ce1-fbc7-475d-92e6-5668e920a4d2-kube-api-access-5698s\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:14:57.592839 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.592834    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba821ce1-fbc7-475d-92e6-5668e920a4d2-extensions-socket-volume\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:14:57.751709 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.751621    2575 generic.go:356] "Generic (PLEG): container finished" podID="ba821ce1-fbc7-475d-92e6-5668e920a4d2" containerID="3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7" exitCode=0
Jul 22 13:14:57.751709 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.751666    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"
Jul 22 13:14:57.751894 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.751701    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq" event={"ID":"ba821ce1-fbc7-475d-92e6-5668e920a4d2","Type":"ContainerDied","Data":"3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7"}
Jul 22 13:14:57.751894 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.751756    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq" event={"ID":"ba821ce1-fbc7-475d-92e6-5668e920a4d2","Type":"ContainerDied","Data":"b43843b4f74fb7fbfc26f4c49fe9d77267c74233a687c53447cf4d377e1453c7"}
Jul 22 13:14:57.751894 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.751779    2575 scope.go:117] "RemoveContainer" containerID="3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7"
Jul 22 13:14:57.767728 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.767710    2575 scope.go:117] "RemoveContainer" containerID="3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7"
Jul 22 13:14:57.768034 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:14:57.768014    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7\": container with ID starting with 3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7 not found: ID does not exist" containerID="3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7"
Jul 22 13:14:57.768110 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.768042    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7"} err="failed to get container status \"3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7\": rpc error: code = NotFound desc = could not find container \"3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7\": container with ID starting with 3dc1815eda7e8d06e1f76e92134d41c34a79520a16b72f636fa7f141b2e5f4d7 not found: ID does not exist"
Jul 22 13:14:57.779027 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.779002    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"]
Jul 22 13:14:57.784409 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:57.784386    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwqbq"]
Jul 22 13:14:58.948765 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:14:58.948732    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba821ce1-fbc7-475d-92e6-5668e920a4d2" path="/var/lib/kubelet/pods/ba821ce1-fbc7-475d-92e6-5668e920a4d2/volumes"
Jul 22 13:15:00.147352 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.147269    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6"]
Jul 22 13:15:00.147789 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.147704    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba821ce1-fbc7-475d-92e6-5668e920a4d2" containerName="manager"
Jul 22 13:15:00.147789 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.147718    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba821ce1-fbc7-475d-92e6-5668e920a4d2" containerName="manager"
Jul 22 13:15:00.147789 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.147747    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a04d8b-b01d-48bb-bea2-6ac57a32d54e" containerName="authorino"
Jul 22 13:15:00.147789 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.147753    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a04d8b-b01d-48bb-bea2-6ac57a32d54e" containerName="authorino"
Jul 22 13:15:00.147951 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.147811    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a04d8b-b01d-48bb-bea2-6ac57a32d54e" containerName="authorino"
Jul 22 13:15:00.147951 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.147826    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba821ce1-fbc7-475d-92e6-5668e920a4d2" containerName="manager"
Jul 22 13:15:00.152270 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.152253    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6"
Jul 22 13:15:00.154640 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.154615    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:15:00.157114 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.157091    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6"]
Jul 22 13:15:00.163999 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.163977    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6"
Jul 22 13:15:00.297506 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.297476    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6"]
Jul 22 13:15:00.299853 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:15:00.299824    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1904046_c101_411e_b3cf_6aa7ae16a0cc.slice/crio-c1d2f0b1435821c29e4ebdf87d08aa612fba1e4a6c9b7eeafc34340665777596 WatchSource:0}: Error finding container c1d2f0b1435821c29e4ebdf87d08aa612fba1e4a6c9b7eeafc34340665777596: Status 404 returned error can't find the container with id c1d2f0b1435821c29e4ebdf87d08aa612fba1e4a6c9b7eeafc34340665777596
Jul 22 13:15:00.301868 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.301850    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 13:15:00.768214 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:00.768181    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6" event={"ID":"c1904046-c101-411e-b3cf-6aa7ae16a0cc","Type":"ContainerStarted","Data":"c1d2f0b1435821c29e4ebdf87d08aa612fba1e4a6c9b7eeafc34340665777596"}
Jul 22 13:15:07.804952 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:07.804898    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6" event={"ID":"c1904046-c101-411e-b3cf-6aa7ae16a0cc","Type":"ContainerStarted","Data":"fa33ae16e9f328d1f127b3185cc709c9d3a62eae4c1dc384dc7bc05200831d00"}
Jul 22 13:15:07.821182 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:07.821137    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6" podStartSLOduration=1.215694986 podStartE2EDuration="7.821121501s" totalImagesPullingTime="6.605426515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:15:07.820323359 +0000 UTC m=+1260.112304394" watchObservedRunningTime="2026-07-22 13:15:07.821121501 +0000 UTC m=+1260.113102535"
Jul 22 13:15:08.810915 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:08.810880    2575 generic.go:356] "Generic (PLEG): container finished" podID="c1904046-c101-411e-b3cf-6aa7ae16a0cc" containerID="fa33ae16e9f328d1f127b3185cc709c9d3a62eae4c1dc384dc7bc05200831d00" exitCode=0
Jul 22 13:15:08.811371 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:08.810954    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6" event={"ID":"c1904046-c101-411e-b3cf-6aa7ae16a0cc","Type":"ContainerDied","Data":"fa33ae16e9f328d1f127b3185cc709c9d3a62eae4c1dc384dc7bc05200831d00"}
Jul 22 13:15:09.948898 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:09.948869    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6"
Jul 22 13:15:10.821218 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:10.821193    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6"
Jul 22 13:15:10.821388 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:10.821187    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745435-ljkw6" event={"ID":"c1904046-c101-411e-b3cf-6aa7ae16a0cc","Type":"ContainerDied","Data":"c1d2f0b1435821c29e4ebdf87d08aa612fba1e4a6c9b7eeafc34340665777596"}
Jul 22 13:15:10.821388 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:15:10.821315    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1d2f0b1435821c29e4ebdf87d08aa612fba1e4a6c9b7eeafc34340665777596"
Jul 22 13:18:40.759923 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.759887    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"]
Jul 22 13:18:40.760500 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.760479    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1904046-c101-411e-b3cf-6aa7ae16a0cc" containerName="cleanup"
Jul 22 13:18:40.760547 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.760503    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1904046-c101-411e-b3cf-6aa7ae16a0cc" containerName="cleanup"
Jul 22 13:18:40.760603 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.760592    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1904046-c101-411e-b3cf-6aa7ae16a0cc" containerName="cleanup"
Jul 22 13:18:40.765087 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.765065    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.767554 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.767527    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-213a9139-kserve-self-signed-certs\""
Jul 22 13:18:40.778427 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.778400    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"]
Jul 22 13:18:40.895630 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.895593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tls-certs\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.895811 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.895640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tmp-dir\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.895811 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.895727    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-home\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.895811 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.895761    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t59wx\" (UniqueName: \"kubernetes.io/projected/0398cfa7-9e4e-4751-8b6b-acd14722eba5-kube-api-access-t59wx\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.895919 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.895816    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-dshm\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.895919 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.895844    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-model-cache\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.996842 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.996804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tls-certs\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.996842 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.996853    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tmp-dir\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.997116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.996908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-home\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.997116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.996944    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t59wx\" (UniqueName: \"kubernetes.io/projected/0398cfa7-9e4e-4751-8b6b-acd14722eba5-kube-api-access-t59wx\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.997116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.997000    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-dshm\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.997116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.997033    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-model-cache\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.997341 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.997284    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-home\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.997341 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.997335    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-model-cache\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.997466 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.997450    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tmp-dir\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.999244 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.999224    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-dshm\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:40.999523 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:40.999505    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tls-certs\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:41.006013 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:41.005992    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t59wx\" (UniqueName: \"kubernetes.io/projected/0398cfa7-9e4e-4751-8b6b-acd14722eba5-kube-api-access-t59wx\") pod \"e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") " pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:41.074788 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:41.074759    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:41.205800 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:41.205775    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"]
Jul 22 13:18:41.208030 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:18:41.208002    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0398cfa7_9e4e_4751_8b6b_acd14722eba5.slice/crio-44145de6fcb3520066de49a4d6e199e4de770652d0fa01adf9229ea376c40126 WatchSource:0}: Error finding container 44145de6fcb3520066de49a4d6e199e4de770652d0fa01adf9229ea376c40126: Status 404 returned error can't find the container with id 44145de6fcb3520066de49a4d6e199e4de770652d0fa01adf9229ea376c40126
Jul 22 13:18:41.767612 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:41.767580    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc" event={"ID":"0398cfa7-9e4e-4751-8b6b-acd14722eba5","Type":"ContainerStarted","Data":"6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c"}
Jul 22 13:18:41.767612 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:41.767617    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc" event={"ID":"0398cfa7-9e4e-4751-8b6b-acd14722eba5","Type":"ContainerStarted","Data":"44145de6fcb3520066de49a4d6e199e4de770652d0fa01adf9229ea376c40126"}
Jul 22 13:18:41.785665 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:41.785617    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc" podStartSLOduration=1.785599777 podStartE2EDuration="1.785599777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:18:41.784893274 +0000 UTC m=+1474.076874308" watchObservedRunningTime="2026-07-22 13:18:41.785599777 +0000 UTC m=+1474.077580809"
Jul 22 13:18:51.075978 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:51.075920    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:51.076360 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:51.076026    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:51.088684 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:51.088657    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:18:51.824631 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:18:51.824600    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:19:08.906199 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:08.906169    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:19:08.909154 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:08.909134    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:19:10.624807 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:10.624773    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"]
Jul 22 13:19:25.645348 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:25.645285    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc" podUID="0398cfa7-9e4e-4751-8b6b-acd14722eba5" containerName="main" containerID="cri-o://6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c" gracePeriod=45
Jul 22 13:19:25.902498 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:25.902434    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:19:25.973786 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:25.973752    2575 generic.go:356] "Generic (PLEG): container finished" podID="0398cfa7-9e4e-4751-8b6b-acd14722eba5" containerID="6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c" exitCode=0
Jul 22 13:19:25.973963 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:25.973794    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc" event={"ID":"0398cfa7-9e4e-4751-8b6b-acd14722eba5","Type":"ContainerDied","Data":"6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c"}
Jul 22 13:19:25.973963 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:25.973811    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"
Jul 22 13:19:25.973963 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:25.973820    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc" event={"ID":"0398cfa7-9e4e-4751-8b6b-acd14722eba5","Type":"ContainerDied","Data":"44145de6fcb3520066de49a4d6e199e4de770652d0fa01adf9229ea376c40126"}
Jul 22 13:19:25.973963 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:25.973836    2575 scope.go:117] "RemoveContainer" containerID="6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c"
Jul 22 13:19:25.982369 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:25.982350    2575 scope.go:117] "RemoveContainer" containerID="6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c"
Jul 22 13:19:25.982639 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:19:25.982612    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c\": container with ID starting with 6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c not found: ID does not exist" containerID="6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c"
Jul 22 13:19:25.982684 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:25.982648    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c"} err="failed to get container status \"6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c\": rpc error: code = NotFound desc = could not find container \"6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c\": container with ID starting with 6c67d42ced508b6d6cf9e4ee5eb075c1b39a66ff032714ec9bd6ea03e8c7e74c not found: ID does not exist"
Jul 22 13:19:26.016618 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.016580    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-dshm\") pod \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") "
Jul 22 13:19:26.016618 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.016620    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-model-cache\") pod \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") "
Jul 22 13:19:26.016853 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.016639    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-home\") pod \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") "
Jul 22 13:19:26.016853 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.016657    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t59wx\" (UniqueName: \"kubernetes.io/projected/0398cfa7-9e4e-4751-8b6b-acd14722eba5-kube-api-access-t59wx\") pod \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") "
Jul 22 13:19:26.016853 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.016690    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tls-certs\") pod \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") "
Jul 22 13:19:26.016853 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.016709    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tmp-dir\") pod \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\" (UID: \"0398cfa7-9e4e-4751-8b6b-acd14722eba5\") "
Jul 22 13:19:26.017070 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.016893    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-model-cache" (OuterVolumeSpecName: "model-cache") pod "0398cfa7-9e4e-4751-8b6b-acd14722eba5" (UID: "0398cfa7-9e4e-4751-8b6b-acd14722eba5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:19:26.017070 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.016908    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-home" (OuterVolumeSpecName: "home") pod "0398cfa7-9e4e-4751-8b6b-acd14722eba5" (UID: "0398cfa7-9e4e-4751-8b6b-acd14722eba5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:19:26.017170 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.017112    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0398cfa7-9e4e-4751-8b6b-acd14722eba5" (UID: "0398cfa7-9e4e-4751-8b6b-acd14722eba5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:19:26.017170 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.017126    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-model-cache\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:19:26.017170 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.017166    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-home\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:19:26.018942 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.018912    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-dshm" (OuterVolumeSpecName: "dshm") pod "0398cfa7-9e4e-4751-8b6b-acd14722eba5" (UID: "0398cfa7-9e4e-4751-8b6b-acd14722eba5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:19:26.019212 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.019193    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0398cfa7-9e4e-4751-8b6b-acd14722eba5-kube-api-access-t59wx" (OuterVolumeSpecName: "kube-api-access-t59wx") pod "0398cfa7-9e4e-4751-8b6b-acd14722eba5" (UID: "0398cfa7-9e4e-4751-8b6b-acd14722eba5"). InnerVolumeSpecName "kube-api-access-t59wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:19:26.019212 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.019195    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0398cfa7-9e4e-4751-8b6b-acd14722eba5" (UID: "0398cfa7-9e4e-4751-8b6b-acd14722eba5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:19:26.118602 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.118563    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tls-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:19:26.118602 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.118593    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-tmp-dir\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:19:26.118602 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.118602    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0398cfa7-9e4e-4751-8b6b-acd14722eba5-dshm\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:19:26.118602 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.118612    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t59wx\" (UniqueName: \"kubernetes.io/projected/0398cfa7-9e4e-4751-8b6b-acd14722eba5-kube-api-access-t59wx\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:19:26.299604 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.299562    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"]
Jul 22 13:19:26.303299 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.303269    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-213a9139-kserve-749ff4dfcf-xdsxc"]
Jul 22 13:19:26.948549 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:19:26.948516    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0398cfa7-9e4e-4751-8b6b-acd14722eba5" path="/var/lib/kubelet/pods/0398cfa7-9e4e-4751-8b6b-acd14722eba5/volumes"
Jul 22 13:22:50.660501 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.660411    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"]
Jul 22 13:22:50.661078 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.660993    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0398cfa7-9e4e-4751-8b6b-acd14722eba5" containerName="main"
Jul 22 13:22:50.661078 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.661009    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0398cfa7-9e4e-4751-8b6b-acd14722eba5" containerName="main"
Jul 22 13:22:50.661190 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.661097    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0398cfa7-9e4e-4751-8b6b-acd14722eba5" containerName="main"
Jul 22 13:22:50.664303 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.664284    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.667501 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.667479    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 13:22:50.668050 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.668017    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2db37d05-serving-cert\""
Jul 22 13:22:50.668050 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.668033    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 13:22:50.668050 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.668019    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5td92\""
Jul 22 13:22:50.672916 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.672894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e805eec7-3839-43f2-8beb-6a04223951f8-openshift-service-ca\") pod \"maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.673033 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.672988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e805eec7-3839-43f2-8beb-6a04223951f8-maas-api-tls\") pod \"maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.673091 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.673033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6m5l\" (UniqueName: \"kubernetes.io/projected/e805eec7-3839-43f2-8beb-6a04223951f8-kube-api-access-t6m5l\") pod \"maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.677609 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.677582    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"]
Jul 22 13:22:50.773593 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.773565    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e805eec7-3839-43f2-8beb-6a04223951f8-maas-api-tls\") pod \"maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.773788 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.773616    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6m5l\" (UniqueName: \"kubernetes.io/projected/e805eec7-3839-43f2-8beb-6a04223951f8-kube-api-access-t6m5l\") pod \"maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.773788 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.773684    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e805eec7-3839-43f2-8beb-6a04223951f8-openshift-service-ca\") pod \"maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.774308 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.774287    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e805eec7-3839-43f2-8beb-6a04223951f8-openshift-service-ca\") pod \"maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.775982 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.775963    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e805eec7-3839-43f2-8beb-6a04223951f8-maas-api-tls\") pod \"maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.782773 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.782753    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6m5l\" (UniqueName: \"kubernetes.io/projected/e805eec7-3839-43f2-8beb-6a04223951f8-kube-api-access-t6m5l\") pod \"maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:50.974528 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:50.974436    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:51.106766 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:51.106742    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"]
Jul 22 13:22:51.108443 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:22:51.108415    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode805eec7_3839_43f2_8beb_6a04223951f8.slice/crio-c0fe388399f22181c36777a40071ba224bdc16580641b3c85bb743a5c1eda2fb WatchSource:0}: Error finding container c0fe388399f22181c36777a40071ba224bdc16580641b3c85bb743a5c1eda2fb: Status 404 returned error can't find the container with id c0fe388399f22181c36777a40071ba224bdc16580641b3c85bb743a5c1eda2fb
Jul 22 13:22:51.109767 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:51.109746    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 13:22:51.926110 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:51.926076    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px" event={"ID":"e805eec7-3839-43f2-8beb-6a04223951f8","Type":"ContainerStarted","Data":"c0fe388399f22181c36777a40071ba224bdc16580641b3c85bb743a5c1eda2fb"}
Jul 22 13:22:54.941304 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:54.941268    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px" event={"ID":"e805eec7-3839-43f2-8beb-6a04223951f8","Type":"ContainerStarted","Data":"4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed"}
Jul 22 13:22:54.941726 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:54.941413    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:22:54.959779 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:22:54.959725    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px" podStartSLOduration=2.127021785 podStartE2EDuration="4.959709974s" totalImagesPullingTime="2.832688189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:22:54.958012807 +0000 UTC m=+1727.249993839" watchObservedRunningTime="2026-07-22 13:22:54.959709974 +0000 UTC m=+1727.251691007"
Jul 22 13:23:00.952317 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:00.952289    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:23:16.134338 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:16.134301    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"]
Jul 22 13:23:16.141233 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:16.141207    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"
Jul 22 13:23:16.142659 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:16.142631    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"]
Jul 22 13:23:16.144004 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:16.143980    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:23:16.196021 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:16.195988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a2923c-a098-4074-b597-42bae859a298-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr\" (UID: \"31a2923c-a098-4074-b597-42bae859a298\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"
Jul 22 13:23:16.296793 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:16.296757    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a2923c-a098-4074-b597-42bae859a298-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr\" (UID: \"31a2923c-a098-4074-b597-42bae859a298\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"
Jul 22 13:23:16.297353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:16.297334    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a2923c-a098-4074-b597-42bae859a298-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr\" (UID: \"31a2923c-a098-4074-b597-42bae859a298\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"
Jul 22 13:23:16.453238 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:16.453143    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"
Jul 22 13:23:16.583497 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:16.583471    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"]
Jul 22 13:23:16.585409 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:23:16.585384    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a2923c_a098_4074_b597_42bae859a298.slice/crio-50f7071156bd98b217fe9d2c6d0464361bf1480f254558be4e76f0d8b5afba62 WatchSource:0}: Error finding container 50f7071156bd98b217fe9d2c6d0464361bf1480f254558be4e76f0d8b5afba62: Status 404 returned error can't find the container with id 50f7071156bd98b217fe9d2c6d0464361bf1480f254558be4e76f0d8b5afba62
Jul 22 13:23:17.036684 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:17.036605    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr" event={"ID":"31a2923c-a098-4074-b597-42bae859a298","Type":"ContainerStarted","Data":"3c027b2869381d68f20b7ae8030db7243efd9426ba68959f9e9c725e01e58a13"}
Jul 22 13:23:17.036684 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:17.036668    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr" event={"ID":"31a2923c-a098-4074-b597-42bae859a298","Type":"ContainerStarted","Data":"50f7071156bd98b217fe9d2c6d0464361bf1480f254558be4e76f0d8b5afba62"}
Jul 22 13:23:17.062014 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:17.061957    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr" podStartSLOduration=1.06192375 podStartE2EDuration="1.06192375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:23:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:23:17.060475745 +0000 UTC m=+1749.352456779" watchObservedRunningTime="2026-07-22 13:23:17.06192375 +0000 UTC m=+1749.353904783"
Jul 22 13:23:18.042223 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:18.042191    2575 generic.go:356] "Generic (PLEG): container finished" podID="31a2923c-a098-4074-b597-42bae859a298" containerID="3c027b2869381d68f20b7ae8030db7243efd9426ba68959f9e9c725e01e58a13" exitCode=0
Jul 22 13:23:18.042660 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:18.042272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr" event={"ID":"31a2923c-a098-4074-b597-42bae859a298","Type":"ContainerDied","Data":"3c027b2869381d68f20b7ae8030db7243efd9426ba68959f9e9c725e01e58a13"}
Jul 22 13:23:19.181518 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:19.181497    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"
Jul 22 13:23:19.325419 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:19.325393    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a2923c-a098-4074-b597-42bae859a298-maas-api-service-ca\") pod \"31a2923c-a098-4074-b597-42bae859a298\" (UID: \"31a2923c-a098-4074-b597-42bae859a298\") "
Jul 22 13:23:19.325791 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:19.325771    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31a2923c-a098-4074-b597-42bae859a298-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "31a2923c-a098-4074-b597-42bae859a298" (UID: "31a2923c-a098-4074-b597-42bae859a298"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:23:19.426561 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:19.426531    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a2923c-a098-4074-b597-42bae859a298-maas-api-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:23:20.052747 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:20.052717    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr" event={"ID":"31a2923c-a098-4074-b597-42bae859a298","Type":"ContainerDied","Data":"50f7071156bd98b217fe9d2c6d0464361bf1480f254558be4e76f0d8b5afba62"}
Jul 22 13:23:20.052747 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:20.052739    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"
Jul 22 13:23:20.052978 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:20.052766    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50f7071156bd98b217fe9d2c6d0464361bf1480f254558be4e76f0d8b5afba62"
Jul 22 13:23:26.559453 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:26.559416    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"]
Jul 22 13:23:26.559986 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:26.559695    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px" podUID="e805eec7-3839-43f2-8beb-6a04223951f8" containerName="maas-api" containerID="cri-o://4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed" gracePeriod=30
Jul 22 13:23:26.805275 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:26.805253    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:23:26.899385 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:26.899338    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e805eec7-3839-43f2-8beb-6a04223951f8-maas-api-tls\") pod \"e805eec7-3839-43f2-8beb-6a04223951f8\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") "
Jul 22 13:23:26.899385 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:26.899387    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6m5l\" (UniqueName: \"kubernetes.io/projected/e805eec7-3839-43f2-8beb-6a04223951f8-kube-api-access-t6m5l\") pod \"e805eec7-3839-43f2-8beb-6a04223951f8\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") "
Jul 22 13:23:26.899612 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:26.899426    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e805eec7-3839-43f2-8beb-6a04223951f8-openshift-service-ca\") pod \"e805eec7-3839-43f2-8beb-6a04223951f8\" (UID: \"e805eec7-3839-43f2-8beb-6a04223951f8\") "
Jul 22 13:23:26.899878 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:26.899839    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e805eec7-3839-43f2-8beb-6a04223951f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e805eec7-3839-43f2-8beb-6a04223951f8" (UID: "e805eec7-3839-43f2-8beb-6a04223951f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:23:26.901519 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:26.901496    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e805eec7-3839-43f2-8beb-6a04223951f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e805eec7-3839-43f2-8beb-6a04223951f8" (UID: "e805eec7-3839-43f2-8beb-6a04223951f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:23:26.901639 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:26.901530    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e805eec7-3839-43f2-8beb-6a04223951f8-kube-api-access-t6m5l" (OuterVolumeSpecName: "kube-api-access-t6m5l") pod "e805eec7-3839-43f2-8beb-6a04223951f8" (UID: "e805eec7-3839-43f2-8beb-6a04223951f8"). InnerVolumeSpecName "kube-api-access-t6m5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:23:27.001147 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.001118    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e805eec7-3839-43f2-8beb-6a04223951f8-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:23:27.001147 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.001147    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6m5l\" (UniqueName: \"kubernetes.io/projected/e805eec7-3839-43f2-8beb-6a04223951f8-kube-api-access-t6m5l\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:23:27.001344 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.001157    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e805eec7-3839-43f2-8beb-6a04223951f8-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:23:27.086954 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.086900    2575 generic.go:356] "Generic (PLEG): container finished" podID="e805eec7-3839-43f2-8beb-6a04223951f8" containerID="4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed" exitCode=0
Jul 22 13:23:27.087116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.086969    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px" event={"ID":"e805eec7-3839-43f2-8beb-6a04223951f8","Type":"ContainerDied","Data":"4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed"}
Jul 22 13:23:27.087116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.087034    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px" event={"ID":"e805eec7-3839-43f2-8beb-6a04223951f8","Type":"ContainerDied","Data":"c0fe388399f22181c36777a40071ba224bdc16580641b3c85bb743a5c1eda2fb"}
Jul 22 13:23:27.087116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.087056    2575 scope.go:117] "RemoveContainer" containerID="4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed"
Jul 22 13:23:27.087116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.086984    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"
Jul 22 13:23:27.095431 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.095411    2575 scope.go:117] "RemoveContainer" containerID="4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed"
Jul 22 13:23:27.095718 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:23:27.095697    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed\": container with ID starting with 4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed not found: ID does not exist" containerID="4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed"
Jul 22 13:23:27.095820 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.095727    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed"} err="failed to get container status \"4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed\": rpc error: code = NotFound desc = could not find container \"4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed\": container with ID starting with 4c1dba515131b0d7afc4a64291cef570bceffe50972a3ac0181e0659d3a353ed not found: ID does not exist"
Jul 22 13:23:27.106349 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.106303    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"]
Jul 22 13:23:27.108200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:27.108174    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2db37d05-75bc746dd8-mj5px"]
Jul 22 13:23:28.947949 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:28.947900    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e805eec7-3839-43f2-8beb-6a04223951f8" path="/var/lib/kubelet/pods/e805eec7-3839-43f2-8beb-6a04223951f8/volumes"
Jul 22 13:23:31.226241 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:31.226207    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"]
Jul 22 13:23:31.228598 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:31.228566    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2db37d05-8hqvr"]
Jul 22 13:23:32.479729 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.479696    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"]
Jul 22 13:23:32.480251 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.480234    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e805eec7-3839-43f2-8beb-6a04223951f8" containerName="maas-api"
Jul 22 13:23:32.480305 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.480254    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e805eec7-3839-43f2-8beb-6a04223951f8" containerName="maas-api"
Jul 22 13:23:32.480305 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.480281    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31a2923c-a098-4074-b597-42bae859a298" containerName="revoke-keys"
Jul 22 13:23:32.480305 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.480291    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="31a2923c-a098-4074-b597-42bae859a298" containerName="revoke-keys"
Jul 22 13:23:32.480408 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.480396    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="31a2923c-a098-4074-b597-42bae859a298" containerName="revoke-keys"
Jul 22 13:23:32.480444 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.480413    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e805eec7-3839-43f2-8beb-6a04223951f8" containerName="maas-api"
Jul 22 13:23:32.486678 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.486656    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.489479 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.489459    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-171700b0-gw-openshift-default-dockercfg-vzdgn\""
Jul 22 13:23:32.490053 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.490033    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 13:23:32.500350 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.500329    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"]
Jul 22 13:23:32.649148 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.649104    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-certs\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.649148 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.649153    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-socket\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.649383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.649179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-envoy\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.649383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.649251    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-credential-socket\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.649383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.649300    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-data\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.649383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.649318    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istiod-ca-cert\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.649383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.649381    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-token\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.649574 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.649422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv8m2\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-kube-api-access-nv8m2\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.649574 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.649464    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-podinfo\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.750990 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.750880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-token\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.750990 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.750972    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv8m2\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-kube-api-access-nv8m2\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751218 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751022    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-podinfo\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751218 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751074    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-certs\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751218 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751101    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-socket\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751218 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-envoy\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751218 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751150    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-credential-socket\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751218 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751177    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-data\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751218 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751204    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istiod-ca-cert\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751669 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751596    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-socket\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751669 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751606    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-certs\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751870 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-credential-socket\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.751870 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751764    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-data\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.752021 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.751995    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istiod-ca-cert\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.753570 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.753543    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-envoy\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.753649 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.753629    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-podinfo\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.762295 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.762276    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv8m2\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-kube-api-access-nv8m2\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.762413 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.762397    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-token\") pod \"e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") " pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.799379 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.799345    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:32.938041 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.935863    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"]
Jul 22 13:23:32.948331 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:32.948304    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31a2923c-a098-4074-b597-42bae859a298" path="/var/lib/kubelet/pods/31a2923c-a098-4074-b597-42bae859a298/volumes"
Jul 22 13:23:33.116659 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:33.116618    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" event={"ID":"b405f7e7-c663-4a01-804c-c3bb6fb8e115","Type":"ContainerStarted","Data":"f6d231d00dcbe8a5a882f1c9d1fb427725f8eb4b895128387b3677277f356b56"}
Jul 22 13:23:35.480324 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:35.480283    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:23:35.480718 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:35.480376    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:23:35.480718 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:35.480428    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:23:36.135832 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:36.135796    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" event={"ID":"b405f7e7-c663-4a01-804c-c3bb6fb8e115","Type":"ContainerStarted","Data":"e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963"}
Jul 22 13:23:36.159393 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:36.159341    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" podStartSLOduration=1.619383096 podStartE2EDuration="4.159323633s" totalImagesPullingTime="2.539940537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:23:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:23:36.157348846 +0000 UTC m=+1768.449329892" watchObservedRunningTime="2026-07-22 13:23:36.159323633 +0000 UTC m=+1768.451304740"
Jul 22 13:23:36.800002 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:36.799966    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:36.805082 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:36.805052    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:37.140785 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.140754    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:37.142337 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.142312    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:37.747057 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.747027    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"]
Jul 22 13:23:37.756239 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.756203    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:37.759836 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.759805    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 13:23:37.760020 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.759854    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-171700b0-serving-cert\""
Jul 22 13:23:37.760020 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.759809    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 13:23:37.760751 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.760732    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5td92\""
Jul 22 13:23:37.761888 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.761862    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"]
Jul 22 13:23:37.800787 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.800751    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f15f4a8-4868-4e36-b766-7e38bbedd013-maas-api-tls\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:37.801167 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.800805    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7js6\" (UniqueName: \"kubernetes.io/projected/4f15f4a8-4868-4e36-b766-7e38bbedd013-kube-api-access-g7js6\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:37.801167 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.800841    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f15f4a8-4868-4e36-b766-7e38bbedd013-openshift-service-ca\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:37.902347 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.902309    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f15f4a8-4868-4e36-b766-7e38bbedd013-maas-api-tls\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:37.902519 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.902379    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7js6\" (UniqueName: \"kubernetes.io/projected/4f15f4a8-4868-4e36-b766-7e38bbedd013-kube-api-access-g7js6\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:37.902519 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.902421    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f15f4a8-4868-4e36-b766-7e38bbedd013-openshift-service-ca\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:37.902519 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:23:37.902451    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-serving-cert: secret "maas-api-e2e-migrate-171700b0-serving-cert" not found
Jul 22 13:23:37.902642 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:23:37.902524    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f15f4a8-4868-4e36-b766-7e38bbedd013-maas-api-tls podName:4f15f4a8-4868-4e36-b766-7e38bbedd013 nodeName:}" failed. No retries permitted until 2026-07-22 13:23:38.402507476 +0000 UTC m=+1770.694488488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4f15f4a8-4868-4e36-b766-7e38bbedd013-maas-api-tls") pod "maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv" (UID: "4f15f4a8-4868-4e36-b766-7e38bbedd013") : secret "maas-api-e2e-migrate-171700b0-serving-cert" not found
Jul 22 13:23:37.903238 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.903218    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f15f4a8-4868-4e36-b766-7e38bbedd013-openshift-service-ca\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:37.912755 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:37.912730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7js6\" (UniqueName: \"kubernetes.io/projected/4f15f4a8-4868-4e36-b766-7e38bbedd013-kube-api-access-g7js6\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:38.409116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:38.409084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f15f4a8-4868-4e36-b766-7e38bbedd013-maas-api-tls\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:38.411514 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:38.411491    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f15f4a8-4868-4e36-b766-7e38bbedd013-maas-api-tls\") pod \"maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:38.669161 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:38.669059    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:38.803013 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:38.802987    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"]
Jul 22 13:23:38.805077 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:23:38.805049    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f15f4a8_4868_4e36_b766_7e38bbedd013.slice/crio-641172344f37dc70ed5fbb06578a233be472ca48f1e8a69edff3bf98b60da979 WatchSource:0}: Error finding container 641172344f37dc70ed5fbb06578a233be472ca48f1e8a69edff3bf98b60da979: Status 404 returned error can't find the container with id 641172344f37dc70ed5fbb06578a233be472ca48f1e8a69edff3bf98b60da979
Jul 22 13:23:39.150636 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:39.150601    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv" event={"ID":"4f15f4a8-4868-4e36-b766-7e38bbedd013","Type":"ContainerStarted","Data":"641172344f37dc70ed5fbb06578a233be472ca48f1e8a69edff3bf98b60da979"}
Jul 22 13:23:40.156362 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:40.156330    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv" event={"ID":"4f15f4a8-4868-4e36-b766-7e38bbedd013","Type":"ContainerStarted","Data":"bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a"}
Jul 22 13:23:40.156777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:40.156478    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:40.175800 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:40.175748    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv" podStartSLOduration=2.007583482 podStartE2EDuration="3.175734427s" totalImagesPullingTime="1.168150945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:23:40.174522339 +0000 UTC m=+1772.466503396" watchObservedRunningTime="2026-07-22 13:23:40.175734427 +0000 UTC m=+1772.467715459"
Jul 22 13:23:46.166565 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:46.166537    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:23:50.207107 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:50.207075    2575 generic.go:356] "Generic (PLEG): container finished" podID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerID="e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963" exitCode=0
Jul 22 13:23:50.207511 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:50.207127    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" event={"ID":"b405f7e7-c663-4a01-804c-c3bb6fb8e115","Type":"ContainerDied","Data":"e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963"}
Jul 22 13:23:50.207554 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:50.207526    2575 scope.go:117] "RemoveContainer" containerID="e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963"
Jul 22 13:23:50.208534 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:50.208505    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:23:50.208595 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:50.208581    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:23:50.208634 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:50.208615    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:23:51.214355 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:51.214321    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" event={"ID":"b405f7e7-c663-4a01-804c-c3bb6fb8e115","Type":"ContainerStarted","Data":"2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9"}
Jul 22 13:23:51.799587 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:51.799550    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:51.800574 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:51.800550    2575 patch_prober.go:28] interesting pod/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 22 13:23:51.800682 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:51.800595    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 22 13:23:52.141292 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:52.141254    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:52.800274 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:52.800240    2575 patch_prober.go:28] interesting pod/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 22 13:23:52.800684 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:52.800291    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 22 13:23:53.800640 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:53.800606    2575 patch_prober.go:28] interesting pod/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 22 13:23:53.801052 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:53.800658    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 22 13:23:54.800300 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:54.800261    2575 patch_prober.go:28] interesting pod/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 22 13:23:54.800502 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:54.800318    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 22 13:23:55.804008 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:55.803979    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:23:55.805124 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:23:55.805107    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:24:02.622378 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:02.622292    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"]
Jul 22 13:24:02.632371 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:02.632337    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"
Jul 22 13:24:02.633434 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:02.633407    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"]
Jul 22 13:24:02.634915 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:02.634890    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:24:02.744493 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:02.744433    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a93e354-60d9-48b3-b730-9bd70136ac06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l\" (UID: \"2a93e354-60d9-48b3-b730-9bd70136ac06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"
Jul 22 13:24:02.845732 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:02.845690    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a93e354-60d9-48b3-b730-9bd70136ac06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l\" (UID: \"2a93e354-60d9-48b3-b730-9bd70136ac06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"
Jul 22 13:24:02.846328 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:02.846309    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a93e354-60d9-48b3-b730-9bd70136ac06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l\" (UID: \"2a93e354-60d9-48b3-b730-9bd70136ac06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"
Jul 22 13:24:02.943974 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:02.943871    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"
Jul 22 13:24:03.282866 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:03.282845    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"]
Jul 22 13:24:03.284810 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:24:03.284772    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a93e354_60d9_48b3_b730_9bd70136ac06.slice/crio-baa93d7fc5ab038ddd7c47ff92d58916549440404398cf5eae52c895f40814d9 WatchSource:0}: Error finding container baa93d7fc5ab038ddd7c47ff92d58916549440404398cf5eae52c895f40814d9: Status 404 returned error can't find the container with id baa93d7fc5ab038ddd7c47ff92d58916549440404398cf5eae52c895f40814d9
Jul 22 13:24:04.282611 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:04.282515    2575 generic.go:356] "Generic (PLEG): container finished" podID="2a93e354-60d9-48b3-b730-9bd70136ac06" containerID="0cf986a479533edec70dcb6fdef907001663e015a0a50d03dd1ec2e67f159761" exitCode=0
Jul 22 13:24:04.282996 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:04.282601    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l" event={"ID":"2a93e354-60d9-48b3-b730-9bd70136ac06","Type":"ContainerDied","Data":"0cf986a479533edec70dcb6fdef907001663e015a0a50d03dd1ec2e67f159761"}
Jul 22 13:24:04.282996 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:04.282664    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l" event={"ID":"2a93e354-60d9-48b3-b730-9bd70136ac06","Type":"ContainerStarted","Data":"baa93d7fc5ab038ddd7c47ff92d58916549440404398cf5eae52c895f40814d9"}
Jul 22 13:24:05.425357 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:05.425332    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"
Jul 22 13:24:05.572528 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:05.572500    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a93e354-60d9-48b3-b730-9bd70136ac06-maas-api-service-ca\") pod \"2a93e354-60d9-48b3-b730-9bd70136ac06\" (UID: \"2a93e354-60d9-48b3-b730-9bd70136ac06\") "
Jul 22 13:24:05.572838 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:05.572818    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a93e354-60d9-48b3-b730-9bd70136ac06-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2a93e354-60d9-48b3-b730-9bd70136ac06" (UID: "2a93e354-60d9-48b3-b730-9bd70136ac06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:24:05.673785 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:05.673747    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a93e354-60d9-48b3-b730-9bd70136ac06-maas-api-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:06.293134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:06.293096    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l" event={"ID":"2a93e354-60d9-48b3-b730-9bd70136ac06","Type":"ContainerDied","Data":"baa93d7fc5ab038ddd7c47ff92d58916549440404398cf5eae52c895f40814d9"}
Jul 22 13:24:06.293293 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:06.293175    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="baa93d7fc5ab038ddd7c47ff92d58916549440404398cf5eae52c895f40814d9"
Jul 22 13:24:06.293293 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:06.293106    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"
Jul 22 13:24:08.935006 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:08.934977    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:24:08.939387 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:08.939367    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:24:13.067836 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.067801    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"]
Jul 22 13:24:13.068316 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.068028    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv" podUID="4f15f4a8-4868-4e36-b766-7e38bbedd013" containerName="maas-api" containerID="cri-o://bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a" gracePeriod=30
Jul 22 13:24:13.320036 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.319969    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:24:13.326885 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.326860    2575 generic.go:356] "Generic (PLEG): container finished" podID="4f15f4a8-4868-4e36-b766-7e38bbedd013" containerID="bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a" exitCode=0
Jul 22 13:24:13.327032 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.326900    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"
Jul 22 13:24:13.327032 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.326951    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv" event={"ID":"4f15f4a8-4868-4e36-b766-7e38bbedd013","Type":"ContainerDied","Data":"bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a"}
Jul 22 13:24:13.327032 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.327003    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv" event={"ID":"4f15f4a8-4868-4e36-b766-7e38bbedd013","Type":"ContainerDied","Data":"641172344f37dc70ed5fbb06578a233be472ca48f1e8a69edff3bf98b60da979"}
Jul 22 13:24:13.327032 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.327028    2575 scope.go:117] "RemoveContainer" containerID="bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a"
Jul 22 13:24:13.335069 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.335031    2575 scope.go:117] "RemoveContainer" containerID="bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a"
Jul 22 13:24:13.335341 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:24:13.335321    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a\": container with ID starting with bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a not found: ID does not exist" containerID="bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a"
Jul 22 13:24:13.335400 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.335351    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a"} err="failed to get container status \"bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a\": rpc error: code = NotFound desc = could not find container \"bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a\": container with ID starting with bbea030052cc25e2169ccc16abbd0ddd0abe32d137a654199978b9f38dcb739a not found: ID does not exist"
Jul 22 13:24:13.440848 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.440822    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f15f4a8-4868-4e36-b766-7e38bbedd013-openshift-service-ca\") pod \"4f15f4a8-4868-4e36-b766-7e38bbedd013\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") "
Jul 22 13:24:13.441060 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.440866    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f15f4a8-4868-4e36-b766-7e38bbedd013-maas-api-tls\") pod \"4f15f4a8-4868-4e36-b766-7e38bbedd013\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") "
Jul 22 13:24:13.441060 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.440898    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7js6\" (UniqueName: \"kubernetes.io/projected/4f15f4a8-4868-4e36-b766-7e38bbedd013-kube-api-access-g7js6\") pod \"4f15f4a8-4868-4e36-b766-7e38bbedd013\" (UID: \"4f15f4a8-4868-4e36-b766-7e38bbedd013\") "
Jul 22 13:24:13.441204 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.441183    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f15f4a8-4868-4e36-b766-7e38bbedd013-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f15f4a8-4868-4e36-b766-7e38bbedd013" (UID: "4f15f4a8-4868-4e36-b766-7e38bbedd013"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:24:13.443007 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.442976    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f15f4a8-4868-4e36-b766-7e38bbedd013-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f15f4a8-4868-4e36-b766-7e38bbedd013" (UID: "4f15f4a8-4868-4e36-b766-7e38bbedd013"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:24:13.443108 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.443010    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f15f4a8-4868-4e36-b766-7e38bbedd013-kube-api-access-g7js6" (OuterVolumeSpecName: "kube-api-access-g7js6") pod "4f15f4a8-4868-4e36-b766-7e38bbedd013" (UID: "4f15f4a8-4868-4e36-b766-7e38bbedd013"). InnerVolumeSpecName "kube-api-access-g7js6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:24:13.541702 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.541670    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f15f4a8-4868-4e36-b766-7e38bbedd013-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:13.541702 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.541697    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f15f4a8-4868-4e36-b766-7e38bbedd013-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:13.541702 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.541707    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7js6\" (UniqueName: \"kubernetes.io/projected/4f15f4a8-4868-4e36-b766-7e38bbedd013-kube-api-access-g7js6\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:13.652216 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.652183    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"]
Jul 22 13:24:13.656596 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:13.656569    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-171700b0-75d8f96d54-fbjcv"]
Jul 22 13:24:14.948558 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:14.948523    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f15f4a8-4868-4e36-b766-7e38bbedd013" path="/var/lib/kubelet/pods/4f15f4a8-4868-4e36-b766-7e38bbedd013/volumes"
Jul 22 13:24:17.703955 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:17.703871    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"]
Jul 22 13:24:17.705874 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:17.705844    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-171700b0-6lg8l"]
Jul 22 13:24:17.840956 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:17.840899    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"]
Jul 22 13:24:17.841226 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:17.841168    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy" containerID="cri-o://2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9" gracePeriod=30
Jul 22 13:24:18.947509 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:18.947473    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a93e354-60d9-48b3-b730-9bd70136ac06" path="/var/lib/kubelet/pods/2a93e354-60d9-48b3-b730-9bd70136ac06/volumes"
Jul 22 13:24:21.249603 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.249566    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"]
Jul 22 13:24:21.250097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.249964    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f15f4a8-4868-4e36-b766-7e38bbedd013" containerName="maas-api"
Jul 22 13:24:21.250097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.249979    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f15f4a8-4868-4e36-b766-7e38bbedd013" containerName="maas-api"
Jul 22 13:24:21.250097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.249987    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a93e354-60d9-48b3-b730-9bd70136ac06" containerName="revoke-keys"
Jul 22 13:24:21.250097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.249993    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a93e354-60d9-48b3-b730-9bd70136ac06" containerName="revoke-keys"
Jul 22 13:24:21.250097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.250045    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f15f4a8-4868-4e36-b766-7e38bbedd013" containerName="maas-api"
Jul 22 13:24:21.250097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.250054    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a93e354-60d9-48b3-b730-9bd70136ac06" containerName="revoke-keys"
Jul 22 13:24:21.255144 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.255117    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.257828 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.257802    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4cedc07b-serving-cert\""
Jul 22 13:24:21.258159 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.258135    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 13:24:21.258913 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.258895    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 13:24:21.259040 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.258900    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5td92\""
Jul 22 13:24:21.264362 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.264338    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"]
Jul 22 13:24:21.313271 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.313230    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5f8116-4f31-4f27-895a-087e7cbf2509-openshift-service-ca\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.313271 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.313273    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5f8116-4f31-4f27-895a-087e7cbf2509-maas-api-tls\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.313480 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.313332    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64gxz\" (UniqueName: \"kubernetes.io/projected/3c5f8116-4f31-4f27-895a-087e7cbf2509-kube-api-access-64gxz\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.414577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.414537    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5f8116-4f31-4f27-895a-087e7cbf2509-openshift-service-ca\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.414577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.414581    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5f8116-4f31-4f27-895a-087e7cbf2509-maas-api-tls\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.414863 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.414605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64gxz\" (UniqueName: \"kubernetes.io/projected/3c5f8116-4f31-4f27-895a-087e7cbf2509-kube-api-access-64gxz\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.414863 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:24:21.414707    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-serving-cert: secret "maas-api-e2e-ait-4cedc07b-serving-cert" not found
Jul 22 13:24:21.414863 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:24:21.414783    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5f8116-4f31-4f27-895a-087e7cbf2509-maas-api-tls podName:3c5f8116-4f31-4f27-895a-087e7cbf2509 nodeName:}" failed. No retries permitted until 2026-07-22 13:24:21.914761921 +0000 UTC m=+1814.206742940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c5f8116-4f31-4f27-895a-087e7cbf2509-maas-api-tls") pod "maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l" (UID: "3c5f8116-4f31-4f27-895a-087e7cbf2509") : secret "maas-api-e2e-ait-4cedc07b-serving-cert" not found
Jul 22 13:24:21.415316 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.415261    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5f8116-4f31-4f27-895a-087e7cbf2509-openshift-service-ca\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.429621 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.429597    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64gxz\" (UniqueName: \"kubernetes.io/projected/3c5f8116-4f31-4f27-895a-087e7cbf2509-kube-api-access-64gxz\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.919305 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.919270    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5f8116-4f31-4f27-895a-087e7cbf2509-maas-api-tls\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:21.921850 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:21.921823    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5f8116-4f31-4f27-895a-087e7cbf2509-maas-api-tls\") pod \"maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:22.168856 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:22.168825    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:22.326439 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:22.326415    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"]
Jul 22 13:24:22.328069 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:24:22.328044    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5f8116_4f31_4f27_895a_087e7cbf2509.slice/crio-e7b6e58a0e21d98492b2628205e4566b7c8f2dba5fe4bfa75b62e3b4e5e82cf9 WatchSource:0}: Error finding container e7b6e58a0e21d98492b2628205e4566b7c8f2dba5fe4bfa75b62e3b4e5e82cf9: Status 404 returned error can't find the container with id e7b6e58a0e21d98492b2628205e4566b7c8f2dba5fe4bfa75b62e3b4e5e82cf9
Jul 22 13:24:22.371891 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:22.371858    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l" event={"ID":"3c5f8116-4f31-4f27-895a-087e7cbf2509","Type":"ContainerStarted","Data":"e7b6e58a0e21d98492b2628205e4566b7c8f2dba5fe4bfa75b62e3b4e5e82cf9"}
Jul 22 13:24:23.096917 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.096896    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:24:23.129765 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.129742    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv8m2\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-kube-api-access-nv8m2\") pod \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") "
Jul 22 13:24:23.129860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.129793    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-podinfo\") pod \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") "
Jul 22 13:24:23.129860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.129833    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-data\") pod \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") "
Jul 22 13:24:23.130049 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.129956    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-credential-socket\") pod \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") "
Jul 22 13:24:23.130049 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130026    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-envoy\") pod \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") "
Jul 22 13:24:23.130153 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130050    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-token\") pod \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") "
Jul 22 13:24:23.130153 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130090    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istiod-ca-cert\") pod \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") "
Jul 22 13:24:23.130153 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130125    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-socket\") pod \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") "
Jul 22 13:24:23.130308 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130163    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-certs\") pod \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\" (UID: \"b405f7e7-c663-4a01-804c-c3bb6fb8e115\") "
Jul 22 13:24:23.130510 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130363    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b405f7e7-c663-4a01-804c-c3bb6fb8e115" (UID: "b405f7e7-c663-4a01-804c-c3bb6fb8e115"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:24:23.130583 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130538    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b405f7e7-c663-4a01-804c-c3bb6fb8e115" (UID: "b405f7e7-c663-4a01-804c-c3bb6fb8e115"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:24:23.130645 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130621    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-credential-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:23.130724 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130698    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-data" (OuterVolumeSpecName: "istio-data") pod "b405f7e7-c663-4a01-804c-c3bb6fb8e115" (UID: "b405f7e7-c663-4a01-804c-c3bb6fb8e115"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:24:23.130950 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130803    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b405f7e7-c663-4a01-804c-c3bb6fb8e115" (UID: "b405f7e7-c663-4a01-804c-c3bb6fb8e115"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:24:23.131059 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.130811    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b405f7e7-c663-4a01-804c-c3bb6fb8e115" (UID: "b405f7e7-c663-4a01-804c-c3bb6fb8e115"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:24:23.132895 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.132837    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b405f7e7-c663-4a01-804c-c3bb6fb8e115" (UID: "b405f7e7-c663-4a01-804c-c3bb6fb8e115"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:24:23.132895 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.132855    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-token" (OuterVolumeSpecName: "istio-token") pod "b405f7e7-c663-4a01-804c-c3bb6fb8e115" (UID: "b405f7e7-c663-4a01-804c-c3bb6fb8e115"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:24:23.133847 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.132911    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b405f7e7-c663-4a01-804c-c3bb6fb8e115" (UID: "b405f7e7-c663-4a01-804c-c3bb6fb8e115"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:24:23.140702 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.135155    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-kube-api-access-nv8m2" (OuterVolumeSpecName: "kube-api-access-nv8m2") pod "b405f7e7-c663-4a01-804c-c3bb6fb8e115" (UID: "b405f7e7-c663-4a01-804c-c3bb6fb8e115"). InnerVolumeSpecName "kube-api-access-nv8m2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:24:23.231522 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.231478    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-podinfo\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:23.231522 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.231519    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-data\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:23.231716 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.231533    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-envoy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:23.231716 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.231545    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istio-token\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:23.231716 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.231558    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b405f7e7-c663-4a01-804c-c3bb6fb8e115-istiod-ca-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:23.231716 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.231569    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:23.231716 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.231581    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b405f7e7-c663-4a01-804c-c3bb6fb8e115-workload-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:23.231716 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.231595    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv8m2\" (UniqueName: \"kubernetes.io/projected/b405f7e7-c663-4a01-804c-c3bb6fb8e115-kube-api-access-nv8m2\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:23.377467 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.377367    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l" event={"ID":"3c5f8116-4f31-4f27-895a-087e7cbf2509","Type":"ContainerStarted","Data":"e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338"}
Jul 22 13:24:23.377467 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.377465    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:23.379330 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.379304    2575 generic.go:356] "Generic (PLEG): container finished" podID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerID="2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9" exitCode=0
Jul 22 13:24:23.379440 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.379346    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" event={"ID":"b405f7e7-c663-4a01-804c-c3bb6fb8e115","Type":"ContainerDied","Data":"2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9"}
Jul 22 13:24:23.379440 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.379373    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"
Jul 22 13:24:23.379440 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.379389    2575 scope.go:117] "RemoveContainer" containerID="2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9"
Jul 22 13:24:23.379568 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.379379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz" event={"ID":"b405f7e7-c663-4a01-804c-c3bb6fb8e115","Type":"ContainerDied","Data":"f6d231d00dcbe8a5a882f1c9d1fb427725f8eb4b895128387b3677277f356b56"}
Jul 22 13:24:23.389019 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.388996    2575 scope.go:117] "RemoveContainer" containerID="e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963"
Jul 22 13:24:23.399630 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.399613    2575 scope.go:117] "RemoveContainer" containerID="2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9"
Jul 22 13:24:23.399879 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:24:23.399862    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9\": container with ID starting with 2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9 not found: ID does not exist" containerID="2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9"
Jul 22 13:24:23.399957 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.399886    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9"} err="failed to get container status \"2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9\": rpc error: code = NotFound desc = could not find container \"2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9\": container with ID starting with 2a360470cfa26222fb8aafd1f5f7731aef6c26dbc7f6880409acf7ee8ea0ade9 not found: ID does not exist"
Jul 22 13:24:23.399957 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.399902    2575 scope.go:117] "RemoveContainer" containerID="e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963"
Jul 22 13:24:23.400146 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:24:23.400131    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963\": container with ID starting with e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963 not found: ID does not exist" containerID="e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963"
Jul 22 13:24:23.400191 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.400150    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963"} err="failed to get container status \"e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963\": rpc error: code = NotFound desc = could not find container \"e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963\": container with ID starting with e9d0cf2d4befb310898ae388da9ed4bd49e1268922b55303373e7511f1198963 not found: ID does not exist"
Jul 22 13:24:23.400641 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.400603    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l" podStartSLOduration=1.631338398 podStartE2EDuration="2.400590352s" totalImagesPullingTime="769.251954ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:24:23.398341732 +0000 UTC m=+1815.690322764" watchObservedRunningTime="2026-07-22 13:24:23.400590352 +0000 UTC m=+1815.692571386"
Jul 22 13:24:23.415826 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.415798    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"]
Jul 22 13:24:23.423365 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:23.423339    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-171700b0-gw-openshift-default-8b5ff8679-mlzhz"]
Jul 22 13:24:24.947812 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:24.947776    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" path="/var/lib/kubelet/pods/b405f7e7-c663-4a01-804c-c3bb6fb8e115/volumes"
Jul 22 13:24:29.390941 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:29.390901    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:46.816912 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.816873    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"]
Jul 22 13:24:46.817511 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.817440    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy"
Jul 22 13:24:46.817511 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.817461    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy"
Jul 22 13:24:46.817511 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.817483    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy"
Jul 22 13:24:46.817511 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.817492    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy"
Jul 22 13:24:46.817733 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.817567    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy"
Jul 22 13:24:46.817733 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.817584    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b405f7e7-c663-4a01-804c-c3bb6fb8e115" containerName="istio-proxy"
Jul 22 13:24:46.820467 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.820445    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"
Jul 22 13:24:46.823192 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.823171    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:24:46.828201 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.828175    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"]
Jul 22 13:24:46.940763 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:46.940721    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn\" (UID: \"5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"
Jul 22 13:24:47.041256 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:47.041222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn\" (UID: \"5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"
Jul 22 13:24:47.041826 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:47.041807    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn\" (UID: \"5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"
Jul 22 13:24:47.132364 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:47.132333    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"
Jul 22 13:24:47.272271 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:47.272248    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"]
Jul 22 13:24:47.274112 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:24:47.274074    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d9231a6_9dd2_4a62_a8a3_ec5e2d7cfc48.slice/crio-06f6bbcbc9afbd3534cceaee450642c9068d43eb9b476f279af55457ac79264e WatchSource:0}: Error finding container 06f6bbcbc9afbd3534cceaee450642c9068d43eb9b476f279af55457ac79264e: Status 404 returned error can't find the container with id 06f6bbcbc9afbd3534cceaee450642c9068d43eb9b476f279af55457ac79264e
Jul 22 13:24:47.498731 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:47.498686    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn" event={"ID":"5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48","Type":"ContainerStarted","Data":"06f6bbcbc9afbd3534cceaee450642c9068d43eb9b476f279af55457ac79264e"}
Jul 22 13:24:48.505442 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:48.505410    2575 generic.go:356] "Generic (PLEG): container finished" podID="5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48" containerID="098cb41fcf10ce01c4501af93354d68a5960b353697920637131b93ff219d659" exitCode=0
Jul 22 13:24:48.505808 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:48.505501    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn" event={"ID":"5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48","Type":"ContainerDied","Data":"098cb41fcf10ce01c4501af93354d68a5960b353697920637131b93ff219d659"}
Jul 22 13:24:49.639693 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:49.639669    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"
Jul 22 13:24:49.768755 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:49.768662    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48-maas-api-service-ca\") pod \"5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48\" (UID: \"5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48\") "
Jul 22 13:24:49.769083 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:49.769055    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48" (UID: "5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:24:49.869353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:49.869323    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48-maas-api-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:50.515103 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:50.515069    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"
Jul 22 13:24:50.515103 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:50.515075    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn" event={"ID":"5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48","Type":"ContainerDied","Data":"06f6bbcbc9afbd3534cceaee450642c9068d43eb9b476f279af55457ac79264e"}
Jul 22 13:24:50.515320 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:50.515124    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06f6bbcbc9afbd3534cceaee450642c9068d43eb9b476f279af55457ac79264e"
Jul 22 13:24:57.251531 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.251491    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"]
Jul 22 13:24:57.252022 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.251770    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l" podUID="3c5f8116-4f31-4f27-895a-087e7cbf2509" containerName="maas-api" containerID="cri-o://e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338" gracePeriod=30
Jul 22 13:24:57.491847 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.491817    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:57.549667 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.549583    2575 generic.go:356] "Generic (PLEG): container finished" podID="3c5f8116-4f31-4f27-895a-087e7cbf2509" containerID="e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338" exitCode=0
Jul 22 13:24:57.549667 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.549637    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"
Jul 22 13:24:57.549667 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.549640    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l" event={"ID":"3c5f8116-4f31-4f27-895a-087e7cbf2509","Type":"ContainerDied","Data":"e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338"}
Jul 22 13:24:57.549877 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.549696    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l" event={"ID":"3c5f8116-4f31-4f27-895a-087e7cbf2509","Type":"ContainerDied","Data":"e7b6e58a0e21d98492b2628205e4566b7c8f2dba5fe4bfa75b62e3b4e5e82cf9"}
Jul 22 13:24:57.549877 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.549726    2575 scope.go:117] "RemoveContainer" containerID="e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338"
Jul 22 13:24:57.558233 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.558217    2575 scope.go:117] "RemoveContainer" containerID="e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338"
Jul 22 13:24:57.558508 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:24:57.558488    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338\": container with ID starting with e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338 not found: ID does not exist" containerID="e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338"
Jul 22 13:24:57.558591 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.558520    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338"} err="failed to get container status \"e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338\": rpc error: code = NotFound desc = could not find container \"e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338\": container with ID starting with e5885b041aecbafd809e470cf7700ae69e8afa2a20da089645af2cb1dcc99338 not found: ID does not exist"
Jul 22 13:24:57.641174 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.641139    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5f8116-4f31-4f27-895a-087e7cbf2509-maas-api-tls\") pod \"3c5f8116-4f31-4f27-895a-087e7cbf2509\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") "
Jul 22 13:24:57.641339 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.641248    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64gxz\" (UniqueName: \"kubernetes.io/projected/3c5f8116-4f31-4f27-895a-087e7cbf2509-kube-api-access-64gxz\") pod \"3c5f8116-4f31-4f27-895a-087e7cbf2509\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") "
Jul 22 13:24:57.641339 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.641315    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5f8116-4f31-4f27-895a-087e7cbf2509-openshift-service-ca\") pod \"3c5f8116-4f31-4f27-895a-087e7cbf2509\" (UID: \"3c5f8116-4f31-4f27-895a-087e7cbf2509\") "
Jul 22 13:24:57.641654 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.641631    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c5f8116-4f31-4f27-895a-087e7cbf2509-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c5f8116-4f31-4f27-895a-087e7cbf2509" (UID: "3c5f8116-4f31-4f27-895a-087e7cbf2509"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:24:57.643140 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.643118    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c5f8116-4f31-4f27-895a-087e7cbf2509-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c5f8116-4f31-4f27-895a-087e7cbf2509" (UID: "3c5f8116-4f31-4f27-895a-087e7cbf2509"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:24:57.643429 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.643402    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c5f8116-4f31-4f27-895a-087e7cbf2509-kube-api-access-64gxz" (OuterVolumeSpecName: "kube-api-access-64gxz") pod "3c5f8116-4f31-4f27-895a-087e7cbf2509" (UID: "3c5f8116-4f31-4f27-895a-087e7cbf2509"). InnerVolumeSpecName "kube-api-access-64gxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:24:57.742335 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.742297    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5f8116-4f31-4f27-895a-087e7cbf2509-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:57.742335 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.742328    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5f8116-4f31-4f27-895a-087e7cbf2509-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:57.742335 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.742338    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64gxz\" (UniqueName: \"kubernetes.io/projected/3c5f8116-4f31-4f27-895a-087e7cbf2509-kube-api-access-64gxz\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:24:57.873474 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.873439    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"]
Jul 22 13:24:57.879057 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:57.879029    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4cedc07b-6b758d46d7-8cf4l"]
Jul 22 13:24:58.948777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:24:58.948743    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c5f8116-4f31-4f27-895a-087e7cbf2509" path="/var/lib/kubelet/pods/3c5f8116-4f31-4f27-895a-087e7cbf2509/volumes"
Jul 22 13:25:01.902555 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:01.902519    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"]
Jul 22 13:25:01.906428 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:01.906404    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4cedc07b-jd2tn"]
Jul 22 13:25:02.947979 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:02.947918    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48" path="/var/lib/kubelet/pods/5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48/volumes"
Jul 22 13:25:03.878837 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.878806    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"]
Jul 22 13:25:03.879206 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.879186    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48" containerName="revoke-keys"
Jul 22 13:25:03.879206 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.879203    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48" containerName="revoke-keys"
Jul 22 13:25:03.879437 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.879221    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c5f8116-4f31-4f27-895a-087e7cbf2509" containerName="maas-api"
Jul 22 13:25:03.879437 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.879227    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c5f8116-4f31-4f27-895a-087e7cbf2509" containerName="maas-api"
Jul 22 13:25:03.879437 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.879287    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c5f8116-4f31-4f27-895a-087e7cbf2509" containerName="maas-api"
Jul 22 13:25:03.879437 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.879297    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d9231a6-9dd2-4a62-a8a3-ec5e2d7cfc48" containerName="revoke-keys"
Jul 22 13:25:03.883984 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.883963    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:03.886628 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.886609    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 13:25:03.886628 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.886620    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7d6ede3b-openshift-default-dockercfg-7hwwx\""
Jul 22 13:25:03.898158 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:03.898132    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"]
Jul 22 13:25:04.001344 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.001306    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-data\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.001344 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.001352    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-token\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.001860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.001381    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-envoy\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.001860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.001407    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfm8f\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-kube-api-access-nfm8f\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.001860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.001513    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-certs\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.001860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.001543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b125849f-af8e-4e02-a49d-b83d60301c5c-istiod-ca-cert\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.001860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.001569    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-socket\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.001860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.001596    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-podinfo\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.001860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.001693    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-credential-socket\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.102673 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.102634    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-certs\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.102673 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.102680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b125849f-af8e-4e02-a49d-b83d60301c5c-istiod-ca-cert\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.102961 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.102708    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-socket\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.102961 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.102746    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-podinfo\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.102961 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.102794    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-credential-socket\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.102961 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.102861    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-data\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.102961 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.102883    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-token\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.102961 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.102913    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-envoy\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.103296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.103118    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-certs\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.103296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.103170    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-socket\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.103296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.103212    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfm8f\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-kube-api-access-nfm8f\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.103296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.103230    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-data\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.103457 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.103327    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-credential-socket\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.103820 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.103794    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b125849f-af8e-4e02-a49d-b83d60301c5c-istiod-ca-cert\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.105297 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.105275    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-envoy\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.105581 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.105561    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-podinfo\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.112685 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.112661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-token\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.113242 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.113220    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfm8f\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-kube-api-access-nfm8f\") pod \"e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") " pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.196998 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.196878    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:04.343260 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.343235    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"]
Jul 22 13:25:04.346171 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:25:04.346142    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb125849f_af8e_4e02_a49d_b83d60301c5c.slice/crio-9aafeb7a1304b9d7fe991c0f8e6cdef117765c904c74526c08e606f6c29e4731 WatchSource:0}: Error finding container 9aafeb7a1304b9d7fe991c0f8e6cdef117765c904c74526c08e606f6c29e4731: Status 404 returned error can't find the container with id 9aafeb7a1304b9d7fe991c0f8e6cdef117765c904c74526c08e606f6c29e4731
Jul 22 13:25:04.348549 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.348519    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:25:04.348634 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.348591    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:25:04.348634 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.348621    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:25:04.585688 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.585652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" event={"ID":"b125849f-af8e-4e02-a49d-b83d60301c5c","Type":"ContainerStarted","Data":"bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d"}
Jul 22 13:25:04.585878 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.585713    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" event={"ID":"b125849f-af8e-4e02-a49d-b83d60301c5c","Type":"ContainerStarted","Data":"9aafeb7a1304b9d7fe991c0f8e6cdef117765c904c74526c08e606f6c29e4731"}
Jul 22 13:25:04.608468 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:04.608411    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" podStartSLOduration=1.608381394 podStartE2EDuration="1.608381394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:25:04.607670984 +0000 UTC m=+1856.899652030" watchObservedRunningTime="2026-07-22 13:25:04.608381394 +0000 UTC m=+1856.900362429"
Jul 22 13:25:05.197399 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:05.197346    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:05.202441 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:05.202409    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:05.590790 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:05.590758    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:05.591775 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:05.591755    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:06.051445 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.051357    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"]
Jul 22 13:25:06.054969 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.054949    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.058344 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.058308    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7d6ede3b-serving-cert\""
Jul 22 13:25:06.058483 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.058322    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 13:25:06.059366 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.059347    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5td92\""
Jul 22 13:25:06.059459 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.059415    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 13:25:06.068478 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.068456    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"]
Jul 22 13:25:06.121335 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.121296    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-maas-api-tls\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.121547 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.121348    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gw5s\" (UniqueName: \"kubernetes.io/projected/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-kube-api-access-7gw5s\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.121547 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.121431    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-openshift-service-ca\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.221976 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.221922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-openshift-service-ca\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.222436 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.222010    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-maas-api-tls\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.222436 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.222068    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gw5s\" (UniqueName: \"kubernetes.io/projected/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-kube-api-access-7gw5s\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.222436 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:25:06.222163    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-serving-cert: secret "maas-api-e2e-derive-7d6ede3b-serving-cert" not found
Jul 22 13:25:06.222436 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:25:06.222251    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-maas-api-tls podName:dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff nodeName:}" failed. No retries permitted until 2026-07-22 13:25:06.722227847 +0000 UTC m=+1859.014208858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-maas-api-tls") pod "maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l" (UID: "dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff") : secret "maas-api-e2e-derive-7d6ede3b-serving-cert" not found
Jul 22 13:25:06.222674 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.222623    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-openshift-service-ca\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.231618 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.231592    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gw5s\" (UniqueName: \"kubernetes.io/projected/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-kube-api-access-7gw5s\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.727041 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.727005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-maas-api-tls\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.729377 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.729350    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-maas-api-tls\") pod \"maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:06.967152 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:06.967121    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:07.109820 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:07.109797    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"]
Jul 22 13:25:07.112014 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:25:07.111986    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5f2c33_8209_4d0d_bd13_bb55ae9bdbff.slice/crio-7dc9f0a040cff7b581b5c18258cf3a3f486ad33b69385980df122b8084d1ea23 WatchSource:0}: Error finding container 7dc9f0a040cff7b581b5c18258cf3a3f486ad33b69385980df122b8084d1ea23: Status 404 returned error can't find the container with id 7dc9f0a040cff7b581b5c18258cf3a3f486ad33b69385980df122b8084d1ea23
Jul 22 13:25:07.601992 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:07.601955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l" event={"ID":"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff","Type":"ContainerStarted","Data":"7dc9f0a040cff7b581b5c18258cf3a3f486ad33b69385980df122b8084d1ea23"}
Jul 22 13:25:08.607813 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:08.607774    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l" event={"ID":"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff","Type":"ContainerStarted","Data":"3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e"}
Jul 22 13:25:08.608298 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:08.607903    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:08.628439 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:08.628380    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l" podStartSLOduration=2.030375162 podStartE2EDuration="2.628363286s" totalImagesPullingTime="597.988124ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:25:08.626715362 +0000 UTC m=+1860.918696424" watchObservedRunningTime="2026-07-22 13:25:08.628363286 +0000 UTC m=+1860.920344318"
Jul 22 13:25:14.617310 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:14.617282    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:19.671249 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:19.671213    2575 generic.go:356] "Generic (PLEG): container finished" podID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerID="bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d" exitCode=0
Jul 22 13:25:19.671662 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:19.671279    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" event={"ID":"b125849f-af8e-4e02-a49d-b83d60301c5c","Type":"ContainerDied","Data":"bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d"}
Jul 22 13:25:19.671662 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:19.671626    2575 scope.go:117] "RemoveContainer" containerID="bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d"
Jul 22 13:25:19.672690 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:19.672663    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:25:19.672753 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:19.672732    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:25:19.672789 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:19.672763    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:25:20.591465 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:20.591432    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:20.677761 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:20.677722    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" event={"ID":"b125849f-af8e-4e02-a49d-b83d60301c5c","Type":"ContainerStarted","Data":"eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623"}
Jul 22 13:25:21.198105 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:21.198062    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:21.199348 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:21.199320    2575 patch_prober.go:28] interesting pod/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 22 13:25:21.199469 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:21.199382    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 22 13:25:22.197572 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:22.197537    2575 patch_prober.go:28] interesting pod/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 22 13:25:22.198006 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:22.197593    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 22 13:25:23.197901 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:23.197863    2575 patch_prober.go:28] interesting pod/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 22 13:25:23.198325 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:23.197960    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 22 13:25:24.197612 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:24.197569    2575 patch_prober.go:28] interesting pod/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 22 13:25:24.197802 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:24.197633    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 22 13:25:25.201379 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:25.201346    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:25.201769 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:25.201629    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:25.202403 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:25.202381    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:30.817772 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:30.817682    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"]
Jul 22 13:25:30.820355 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:30.820338    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"
Jul 22 13:25:30.822798 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:30.822776    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:25:30.829099 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:30.829075    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"]
Jul 22 13:25:30.978257 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:30.978223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680a4816-2a90-48e8-9acd-4787bdf91b7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z\" (UID: \"680a4816-2a90-48e8-9acd-4787bdf91b7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"
Jul 22 13:25:31.079724 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:31.079694    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680a4816-2a90-48e8-9acd-4787bdf91b7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z\" (UID: \"680a4816-2a90-48e8-9acd-4787bdf91b7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"
Jul 22 13:25:31.080355 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:31.080335    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680a4816-2a90-48e8-9acd-4787bdf91b7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z\" (UID: \"680a4816-2a90-48e8-9acd-4787bdf91b7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"
Jul 22 13:25:31.131987 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:31.131955    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"
Jul 22 13:25:31.262694 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:31.262670    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"]
Jul 22 13:25:31.264751 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:25:31.264723    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680a4816_2a90_48e8_9acd_4787bdf91b7d.slice/crio-cd4b76393631413fef5f56f49f96b20136e62ebfaa47d2e122936c5d17f8e30c WatchSource:0}: Error finding container cd4b76393631413fef5f56f49f96b20136e62ebfaa47d2e122936c5d17f8e30c: Status 404 returned error can't find the container with id cd4b76393631413fef5f56f49f96b20136e62ebfaa47d2e122936c5d17f8e30c
Jul 22 13:25:31.736190 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:31.736095    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z" event={"ID":"680a4816-2a90-48e8-9acd-4787bdf91b7d","Type":"ContainerStarted","Data":"7639779cb6c8ecde0e7a52e2bde63ce103c51f4f90e8ae80546f0e5f2874734e"}
Jul 22 13:25:31.736190 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:31.736147    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z" event={"ID":"680a4816-2a90-48e8-9acd-4787bdf91b7d","Type":"ContainerStarted","Data":"cd4b76393631413fef5f56f49f96b20136e62ebfaa47d2e122936c5d17f8e30c"}
Jul 22 13:25:31.755335 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:31.755276    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z" podStartSLOduration=1.75526047 podStartE2EDuration="1.75526047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:25:31.753746535 +0000 UTC m=+1884.045727565" watchObservedRunningTime="2026-07-22 13:25:31.75526047 +0000 UTC m=+1884.047241512"
Jul 22 13:25:32.743361 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:32.743325    2575 generic.go:356] "Generic (PLEG): container finished" podID="680a4816-2a90-48e8-9acd-4787bdf91b7d" containerID="7639779cb6c8ecde0e7a52e2bde63ce103c51f4f90e8ae80546f0e5f2874734e" exitCode=0
Jul 22 13:25:32.743742 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:32.743412    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z" event={"ID":"680a4816-2a90-48e8-9acd-4787bdf91b7d","Type":"ContainerDied","Data":"7639779cb6c8ecde0e7a52e2bde63ce103c51f4f90e8ae80546f0e5f2874734e"}
Jul 22 13:25:33.881102 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:33.881071    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"
Jul 22 13:25:34.008445 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:34.008351    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680a4816-2a90-48e8-9acd-4787bdf91b7d-maas-api-service-ca\") pod \"680a4816-2a90-48e8-9acd-4787bdf91b7d\" (UID: \"680a4816-2a90-48e8-9acd-4787bdf91b7d\") "
Jul 22 13:25:34.008771 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:34.008748    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680a4816-2a90-48e8-9acd-4787bdf91b7d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "680a4816-2a90-48e8-9acd-4787bdf91b7d" (UID: "680a4816-2a90-48e8-9acd-4787bdf91b7d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:25:34.109953 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:34.109898    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680a4816-2a90-48e8-9acd-4787bdf91b7d-maas-api-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:34.753734 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:34.753707    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"
Jul 22 13:25:34.753734 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:34.753729    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z" event={"ID":"680a4816-2a90-48e8-9acd-4787bdf91b7d","Type":"ContainerDied","Data":"cd4b76393631413fef5f56f49f96b20136e62ebfaa47d2e122936c5d17f8e30c"}
Jul 22 13:25:34.753966 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:34.753798    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd4b76393631413fef5f56f49f96b20136e62ebfaa47d2e122936c5d17f8e30c"
Jul 22 13:25:41.258671 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.258633    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"]
Jul 22 13:25:41.259061 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.258867    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l" podUID="dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff" containerName="maas-api" containerID="cri-o://3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e" gracePeriod=30
Jul 22 13:25:41.504856 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.504823    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:41.578342 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.578296    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-openshift-service-ca\") pod \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") "
Jul 22 13:25:41.578558 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.578355    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gw5s\" (UniqueName: \"kubernetes.io/projected/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-kube-api-access-7gw5s\") pod \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") "
Jul 22 13:25:41.578558 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.578398    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-maas-api-tls\") pod \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\" (UID: \"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff\") "
Jul 22 13:25:41.578690 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.578660    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff" (UID: "dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:25:41.580498 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.580460    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-kube-api-access-7gw5s" (OuterVolumeSpecName: "kube-api-access-7gw5s") pod "dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff" (UID: "dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff"). InnerVolumeSpecName "kube-api-access-7gw5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:25:41.580610 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.580546    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff" (UID: "dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:25:41.679823 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.679774    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.679823 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.679813    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gw5s\" (UniqueName: \"kubernetes.io/projected/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-kube-api-access-7gw5s\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.679823 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.679826    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.789347 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.789311    2575 generic.go:356] "Generic (PLEG): container finished" podID="dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff" containerID="3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e" exitCode=0
Jul 22 13:25:41.789537 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.789359    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"
Jul 22 13:25:41.789537 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.789390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l" event={"ID":"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff","Type":"ContainerDied","Data":"3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e"}
Jul 22 13:25:41.789537 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.789442    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l" event={"ID":"dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff","Type":"ContainerDied","Data":"7dc9f0a040cff7b581b5c18258cf3a3f486ad33b69385980df122b8084d1ea23"}
Jul 22 13:25:41.789537 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.789464    2575 scope.go:117] "RemoveContainer" containerID="3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e"
Jul 22 13:25:41.799076 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.799054    2575 scope.go:117] "RemoveContainer" containerID="3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e"
Jul 22 13:25:41.799373 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:25:41.799355    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e\": container with ID starting with 3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e not found: ID does not exist" containerID="3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e"
Jul 22 13:25:41.799445 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.799383    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e"} err="failed to get container status \"3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e\": rpc error: code = NotFound desc = could not find container \"3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e\": container with ID starting with 3576ab39e027a7dd5e9c3f7e6cf3bffaa019414cd11ee4dbcf8eda5c2f33bc8e not found: ID does not exist"
Jul 22 13:25:41.813795 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.813764    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"]
Jul 22 13:25:41.820331 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:41.820308    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d6ede3b-66696f5dbb-zf28l"]
Jul 22 13:25:42.948093 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:42.948062    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff" path="/var/lib/kubelet/pods/dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff/volumes"
Jul 22 13:25:45.902841 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:45.902804    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"]
Jul 22 13:25:45.905954 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:45.905908    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d6ede3b-cgn4z"]
Jul 22 13:25:46.056294 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:46.056265    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"]
Jul 22 13:25:46.056509 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:46.056472    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy" containerID="cri-o://eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623" gracePeriod=30
Jul 22 13:25:46.950840 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:46.950806    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="680a4816-2a90-48e8-9acd-4787bdf91b7d" path="/var/lib/kubelet/pods/680a4816-2a90-48e8-9acd-4787bdf91b7d/volumes"
Jul 22 13:25:51.304060 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.304037    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:51.365477 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365449    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-envoy\") pod \"b125849f-af8e-4e02-a49d-b83d60301c5c\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") "
Jul 22 13:25:51.365662 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365487    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-certs\") pod \"b125849f-af8e-4e02-a49d-b83d60301c5c\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") "
Jul 22 13:25:51.365662 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365506    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b125849f-af8e-4e02-a49d-b83d60301c5c-istiod-ca-cert\") pod \"b125849f-af8e-4e02-a49d-b83d60301c5c\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") "
Jul 22 13:25:51.365662 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365527    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-credential-socket\") pod \"b125849f-af8e-4e02-a49d-b83d60301c5c\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") "
Jul 22 13:25:51.365662 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365553    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-podinfo\") pod \"b125849f-af8e-4e02-a49d-b83d60301c5c\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") "
Jul 22 13:25:51.365662 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365652    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-data\") pod \"b125849f-af8e-4e02-a49d-b83d60301c5c\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") "
Jul 22 13:25:51.365957 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365693    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfm8f\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-kube-api-access-nfm8f\") pod \"b125849f-af8e-4e02-a49d-b83d60301c5c\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") "
Jul 22 13:25:51.365957 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365765    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-token\") pod \"b125849f-af8e-4e02-a49d-b83d60301c5c\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") "
Jul 22 13:25:51.365957 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365788    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b125849f-af8e-4e02-a49d-b83d60301c5c" (UID: "b125849f-af8e-4e02-a49d-b83d60301c5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:51.365957 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365801    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b125849f-af8e-4e02-a49d-b83d60301c5c" (UID: "b125849f-af8e-4e02-a49d-b83d60301c5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:51.365957 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365819    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-socket\") pod \"b125849f-af8e-4e02-a49d-b83d60301c5c\" (UID: \"b125849f-af8e-4e02-a49d-b83d60301c5c\") "
Jul 22 13:25:51.365957 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.365884    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b125849f-af8e-4e02-a49d-b83d60301c5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b125849f-af8e-4e02-a49d-b83d60301c5c" (UID: "b125849f-af8e-4e02-a49d-b83d60301c5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:25:51.366307 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.366146    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:51.366307 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.366166    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b125849f-af8e-4e02-a49d-b83d60301c5c-istiod-ca-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:51.366307 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.366181    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-credential-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:51.366469 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.366432    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b125849f-af8e-4e02-a49d-b83d60301c5c" (UID: "b125849f-af8e-4e02-a49d-b83d60301c5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:51.366852 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.366827    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "b125849f-af8e-4e02-a49d-b83d60301c5c" (UID: "b125849f-af8e-4e02-a49d-b83d60301c5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:51.367921 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.367896    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "b125849f-af8e-4e02-a49d-b83d60301c5c" (UID: "b125849f-af8e-4e02-a49d-b83d60301c5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:25:51.368191 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.368162    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b125849f-af8e-4e02-a49d-b83d60301c5c" (UID: "b125849f-af8e-4e02-a49d-b83d60301c5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:51.368191 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.368171    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b125849f-af8e-4e02-a49d-b83d60301c5c" (UID: "b125849f-af8e-4e02-a49d-b83d60301c5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:25:51.368336 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.368252    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-kube-api-access-nfm8f" (OuterVolumeSpecName: "kube-api-access-nfm8f") pod "b125849f-af8e-4e02-a49d-b83d60301c5c" (UID: "b125849f-af8e-4e02-a49d-b83d60301c5c"). InnerVolumeSpecName "kube-api-access-nfm8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:25:51.467470 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.467415    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-envoy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:51.467470 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.467462    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-podinfo\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:51.467470 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.467471    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-data\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:51.467470 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.467481    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfm8f\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-kube-api-access-nfm8f\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:51.467470 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.467490    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b125849f-af8e-4e02-a49d-b83d60301c5c-istio-token\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:51.467788 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.467499    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b125849f-af8e-4e02-a49d-b83d60301c5c-workload-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:25:51.837452 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.837417    2575 generic.go:356] "Generic (PLEG): container finished" podID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerID="eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623" exitCode=0
Jul 22 13:25:51.837669 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.837463    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"
Jul 22 13:25:51.837669 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.837499    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" event={"ID":"b125849f-af8e-4e02-a49d-b83d60301c5c","Type":"ContainerDied","Data":"eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623"}
Jul 22 13:25:51.837669 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.837556    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm" event={"ID":"b125849f-af8e-4e02-a49d-b83d60301c5c","Type":"ContainerDied","Data":"9aafeb7a1304b9d7fe991c0f8e6cdef117765c904c74526c08e606f6c29e4731"}
Jul 22 13:25:51.837669 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.837580    2575 scope.go:117] "RemoveContainer" containerID="eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623"
Jul 22 13:25:51.849164 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.849134    2575 scope.go:117] "RemoveContainer" containerID="bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d"
Jul 22 13:25:51.858743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.858724    2575 scope.go:117] "RemoveContainer" containerID="eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623"
Jul 22 13:25:51.859034 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:25:51.859014    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623\": container with ID starting with eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623 not found: ID does not exist" containerID="eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623"
Jul 22 13:25:51.859094 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.859043    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623"} err="failed to get container status \"eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623\": rpc error: code = NotFound desc = could not find container \"eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623\": container with ID starting with eda87076d7410a431746b5a02d61944705941af00ba0fc60cd1f8cd0edb25623 not found: ID does not exist"
Jul 22 13:25:51.859094 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.859062    2575 scope.go:117] "RemoveContainer" containerID="bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d"
Jul 22 13:25:51.859311 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:25:51.859294    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d\": container with ID starting with bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d not found: ID does not exist" containerID="bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d"
Jul 22 13:25:51.859368 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.859320    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d"} err="failed to get container status \"bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d\": rpc error: code = NotFound desc = could not find container \"bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d\": container with ID starting with bf615bacee3cdf1f3aa006476ecb4db2905f50b8129512ea0339ad7486a5db0d not found: ID does not exist"
Jul 22 13:25:51.867456 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.867431    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"]
Jul 22 13:25:51.872110 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:51.872086    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7d6ede3b-openshift-default-57556fff99-mzjtm"]
Jul 22 13:25:52.948846 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:25:52.948811    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" path="/var/lib/kubelet/pods/b125849f-af8e-4e02-a49d-b83d60301c5c/volumes"
Jul 22 13:27:26.647305 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647225    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"]
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647640    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="680a4816-2a90-48e8-9acd-4787bdf91b7d" containerName="revoke-keys"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647652    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="680a4816-2a90-48e8-9acd-4787bdf91b7d" containerName="revoke-keys"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647663    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff" containerName="maas-api"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647669    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff" containerName="maas-api"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647677    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647684    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647697    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647702    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647762    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="680a4816-2a90-48e8-9acd-4787bdf91b7d" containerName="revoke-keys"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647772    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd5f2c33-8209-4d0d-bd13-bb55ae9bdbff" containerName="maas-api"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647779    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy"
Jul 22 13:27:26.647881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.647786    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b125849f-af8e-4e02-a49d-b83d60301c5c" containerName="istio-proxy"
Jul 22 13:27:26.650894 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.650871    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.660679 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.660647    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-16e9e526-openshift-default-dockercfg-x4k8w\""
Jul 22 13:27:26.661224 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.661177    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 13:27:26.674200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.674172    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"]
Jul 22 13:27:26.721589 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.721547    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-credential-socket\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.721777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.721603    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-istio-token\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.721777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.721674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxdqg\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-kube-api-access-fxdqg\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.721867 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.721832    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-socket\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.721867 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.721860    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-envoy\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.721974 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.721950    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-data\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.722041 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.721990    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d22c0cc0-1339-435e-8979-4285cde324aa-istio-podinfo\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.722041 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.722023    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d22c0cc0-1339-435e-8979-4285cde324aa-istiod-ca-cert\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.722124 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.722104    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-certs\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.823531 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.823500    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-certs\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.823531 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.823539    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-credential-socket\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.823796 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.823568    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-istio-token\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.823796 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.823597    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxdqg\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-kube-api-access-fxdqg\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.823796 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.823715    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-socket\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.824023 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.823961    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-envoy\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.824023 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.824015    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-data\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.824134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.824041    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d22c0cc0-1339-435e-8979-4285cde324aa-istio-podinfo\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.824134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.824067    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d22c0cc0-1339-435e-8979-4285cde324aa-istiod-ca-cert\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.824134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.824067    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-certs\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.824134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.824128    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-credential-socket\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.824416 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.824181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-socket\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.824498 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.824475    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-data\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.824753 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.824730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d22c0cc0-1339-435e-8979-4285cde324aa-istiod-ca-cert\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.826827 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.826801    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-envoy\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.827157 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.827132    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d22c0cc0-1339-435e-8979-4285cde324aa-istio-podinfo\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.837227 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.837199    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxdqg\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-kube-api-access-fxdqg\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.837507 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.837456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-istio-token\") pod \"e2e-mt-16e9e526-openshift-default-58db46887-vrmfq\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") " pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:26.971889 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:26.971799    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:27.155705 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:27.155381    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"]
Jul 22 13:27:27.160665 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:27:27.160633    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22c0cc0_1339_435e_8979_4285cde324aa.slice/crio-adfd1f981093f8bfa5dcaf397ab2c84c1e19f72605684ec8b7e0343abd595d77 WatchSource:0}: Error finding container adfd1f981093f8bfa5dcaf397ab2c84c1e19f72605684ec8b7e0343abd595d77: Status 404 returned error can't find the container with id adfd1f981093f8bfa5dcaf397ab2c84c1e19f72605684ec8b7e0343abd595d77
Jul 22 13:27:27.162769 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:27.162737    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:27:27.162908 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:27.162823    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:27:27.162908 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:27.162871    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:27:27.295407 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:27.295375    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" event={"ID":"d22c0cc0-1339-435e-8979-4285cde324aa","Type":"ContainerStarted","Data":"4486a9b0c3bf162f7d3caf0d86bc6315a6bd65a10fb1e1d124b6f70a9424dc63"}
Jul 22 13:27:27.295614 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:27.295439    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" event={"ID":"d22c0cc0-1339-435e-8979-4285cde324aa","Type":"ContainerStarted","Data":"adfd1f981093f8bfa5dcaf397ab2c84c1e19f72605684ec8b7e0343abd595d77"}
Jul 22 13:27:27.326725 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:27.326642    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" podStartSLOduration=1.3266160390000001 podStartE2EDuration="1.326616039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:27:27.322871712 +0000 UTC m=+1999.614852746" watchObservedRunningTime="2026-07-22 13:27:27.326616039 +0000 UTC m=+1999.618597075"
Jul 22 13:27:27.972783 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:27.972744    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:28.977190 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:28.977162    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:29.305577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.305491    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:29.306577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.306552    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:29.644968 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.644917    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"]
Jul 22 13:27:29.648425 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.648408    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.651864 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.651842    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5td92\""
Jul 22 13:27:29.652307 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.652291    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-16e9e526-serving-cert\""
Jul 22 13:27:29.653039 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.653015    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 13:27:29.660243 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.653558    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 13:27:29.665706 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.665676    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"]
Jul 22 13:27:29.751525 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.751488    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-maas-api-tls\") pod \"maas-api-e2e-mt-16e9e526-59865b4447-cdqqv\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.751686 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.751655    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-openshift-service-ca\") pod \"maas-api-e2e-mt-16e9e526-59865b4447-cdqqv\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.751758 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.751695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwbq6\" (UniqueName: \"kubernetes.io/projected/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-kube-api-access-jwbq6\") pod \"maas-api-e2e-mt-16e9e526-59865b4447-cdqqv\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.852987 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.852916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-openshift-service-ca\") pod \"maas-api-e2e-mt-16e9e526-59865b4447-cdqqv\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.853195 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.853001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwbq6\" (UniqueName: \"kubernetes.io/projected/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-kube-api-access-jwbq6\") pod \"maas-api-e2e-mt-16e9e526-59865b4447-cdqqv\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.853195 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.853023    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-maas-api-tls\") pod \"maas-api-e2e-mt-16e9e526-59865b4447-cdqqv\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.853665 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.853640    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-openshift-service-ca\") pod \"maas-api-e2e-mt-16e9e526-59865b4447-cdqqv\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.855550 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.855527    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-maas-api-tls\") pod \"maas-api-e2e-mt-16e9e526-59865b4447-cdqqv\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.862245 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.862217    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwbq6\" (UniqueName: \"kubernetes.io/projected/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-kube-api-access-jwbq6\") pod \"maas-api-e2e-mt-16e9e526-59865b4447-cdqqv\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:29.967155 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:29.967069    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:30.102300 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:30.102274    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"]
Jul 22 13:27:30.104550 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:27:30.104521    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f8e150f_4bf0_4b25_9b44_5debf5c2958f.slice/crio-4232ba95f37c5b23b934149c82b47bba82101a1744aabfbb698c654d58963d04 WatchSource:0}: Error finding container 4232ba95f37c5b23b934149c82b47bba82101a1744aabfbb698c654d58963d04: Status 404 returned error can't find the container with id 4232ba95f37c5b23b934149c82b47bba82101a1744aabfbb698c654d58963d04
Jul 22 13:27:30.311257 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:30.311175    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv" event={"ID":"6f8e150f-4bf0-4b25-9b44-5debf5c2958f","Type":"ContainerStarted","Data":"4232ba95f37c5b23b934149c82b47bba82101a1744aabfbb698c654d58963d04"}
Jul 22 13:27:31.317569 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:31.317529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv" event={"ID":"6f8e150f-4bf0-4b25-9b44-5debf5c2958f","Type":"ContainerStarted","Data":"9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172"}
Jul 22 13:27:31.317996 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:31.317600    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:31.337939 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:31.337873    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv" podStartSLOduration=1.875412434 podStartE2EDuration="2.337854659s" totalImagesPullingTime="462.442225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:27:31.336652627 +0000 UTC m=+2003.628633672" watchObservedRunningTime="2026-07-22 13:27:31.337854659 +0000 UTC m=+2003.629835693"
Jul 22 13:27:37.333279 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:37.333238    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:41.378474 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.378435    2575 generic.go:356] "Generic (PLEG): container finished" podID="d22c0cc0-1339-435e-8979-4285cde324aa" containerID="4486a9b0c3bf162f7d3caf0d86bc6315a6bd65a10fb1e1d124b6f70a9424dc63" exitCode=0
Jul 22 13:27:41.378905 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.378506    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" event={"ID":"d22c0cc0-1339-435e-8979-4285cde324aa","Type":"ContainerDied","Data":"4486a9b0c3bf162f7d3caf0d86bc6315a6bd65a10fb1e1d124b6f70a9424dc63"}
Jul 22 13:27:41.378905 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.378851    2575 scope.go:117] "RemoveContainer" containerID="4486a9b0c3bf162f7d3caf0d86bc6315a6bd65a10fb1e1d124b6f70a9424dc63"
Jul 22 13:27:41.379831 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.379803    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:27:41.379886 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.379874    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:27:41.379961 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.379921    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:27:41.426458 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.426427    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"]
Jul 22 13:27:41.431730 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.431712    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"
Jul 22 13:27:41.434334 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.434315    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:27:41.437983 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.436486    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"]
Jul 22 13:27:41.563663 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.563625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2\" (UID: \"32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"
Jul 22 13:27:41.664864 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.664771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2\" (UID: \"32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"
Jul 22 13:27:41.665448 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.665423    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2\" (UID: \"32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"
Jul 22 13:27:41.748663 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.748617    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"
Jul 22 13:27:41.880768 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:41.880743    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"]
Jul 22 13:27:41.883281 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:27:41.883248    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a4d2a4_9b6e_4b7d_b13b_0d4be8ca576f.slice/crio-fa83eae845819aeeeae31ffc47595a6bee9d9a3b4aaa9eaddb610be6f97554bb WatchSource:0}: Error finding container fa83eae845819aeeeae31ffc47595a6bee9d9a3b4aaa9eaddb610be6f97554bb: Status 404 returned error can't find the container with id fa83eae845819aeeeae31ffc47595a6bee9d9a3b4aaa9eaddb610be6f97554bb
Jul 22 13:27:42.385512 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:42.385470    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" event={"ID":"d22c0cc0-1339-435e-8979-4285cde324aa","Type":"ContainerStarted","Data":"73fc3b73d6153a8f02470dff3aa36a5f179d54207f8eb47ce00b668cf5bc7d04"}
Jul 22 13:27:42.386959 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:42.386921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2" event={"ID":"32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f","Type":"ContainerStarted","Data":"64d996be5ab16ce67cc8c1614aca3c5eebcef09d8a4dbf048ecc6dc2a52314f6"}
Jul 22 13:27:42.387066 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:42.386982    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2" event={"ID":"32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f","Type":"ContainerStarted","Data":"fa83eae845819aeeeae31ffc47595a6bee9d9a3b4aaa9eaddb610be6f97554bb"}
Jul 22 13:27:42.423993 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:42.423942    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2" podStartSLOduration=1.423911496 podStartE2EDuration="1.423911496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:27:42.422453224 +0000 UTC m=+2014.714434258" watchObservedRunningTime="2026-07-22 13:27:42.423911496 +0000 UTC m=+2014.715892565"
Jul 22 13:27:42.972580 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:42.972489    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:42.973667 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:42.973638    2575 patch_prober.go:28] interesting pod/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 22 13:27:42.973791 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:42.973690    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 22 13:27:43.393042 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:43.393012    2575 generic.go:356] "Generic (PLEG): container finished" podID="32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f" containerID="64d996be5ab16ce67cc8c1614aca3c5eebcef09d8a4dbf048ecc6dc2a52314f6" exitCode=0
Jul 22 13:27:43.393450 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:43.393107    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2" event={"ID":"32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f","Type":"ContainerDied","Data":"64d996be5ab16ce67cc8c1614aca3c5eebcef09d8a4dbf048ecc6dc2a52314f6"}
Jul 22 13:27:43.972650 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:43.972613    2575 patch_prober.go:28] interesting pod/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 22 13:27:43.972851 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:43.972668    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 22 13:27:44.306063 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:44.305979    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:44.571034 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:44.571012    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"
Jul 22 13:27:44.695877 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:44.695841    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f-maas-api-service-ca\") pod \"32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f\" (UID: \"32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f\") "
Jul 22 13:27:44.696184 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:44.696162    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f" (UID: "32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:27:44.796948 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:44.796897    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f-maas-api-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:27:44.972942 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:44.972904    2575 patch_prober.go:28] interesting pod/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 22 13:27:44.973095 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:44.972973    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 22 13:27:45.404965 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:45.404535    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2" event={"ID":"32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f","Type":"ContainerDied","Data":"fa83eae845819aeeeae31ffc47595a6bee9d9a3b4aaa9eaddb610be6f97554bb"}
Jul 22 13:27:45.404965 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:45.404596    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa83eae845819aeeeae31ffc47595a6bee9d9a3b4aaa9eaddb610be6f97554bb"
Jul 22 13:27:45.404965 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:45.404657    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"
Jul 22 13:27:45.973207 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:45.973166    2575 patch_prober.go:28] interesting pod/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 22 13:27:45.973607 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:45.973230    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 22 13:27:46.976085 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:46.976055    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:46.977039 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:46.977021    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:27:51.993452 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:51.993418    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"]
Jul 22 13:27:51.993911 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:51.993623    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv" podUID="6f8e150f-4bf0-4b25-9b44-5debf5c2958f" containerName="maas-api" containerID="cri-o://9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172" gracePeriod=30
Jul 22 13:27:52.248416 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.248352    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:52.363787 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.363753    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-openshift-service-ca\") pod \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") "
Jul 22 13:27:52.363991 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.363852    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwbq6\" (UniqueName: \"kubernetes.io/projected/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-kube-api-access-jwbq6\") pod \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") "
Jul 22 13:27:52.363991 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.363873    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-maas-api-tls\") pod \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\" (UID: \"6f8e150f-4bf0-4b25-9b44-5debf5c2958f\") "
Jul 22 13:27:52.364120 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.364100    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f8e150f-4bf0-4b25-9b44-5debf5c2958f" (UID: "6f8e150f-4bf0-4b25-9b44-5debf5c2958f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:27:52.365940 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.365904    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f8e150f-4bf0-4b25-9b44-5debf5c2958f" (UID: "6f8e150f-4bf0-4b25-9b44-5debf5c2958f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:27:52.365940 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.365918    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-kube-api-access-jwbq6" (OuterVolumeSpecName: "kube-api-access-jwbq6") pod "6f8e150f-4bf0-4b25-9b44-5debf5c2958f" (UID: "6f8e150f-4bf0-4b25-9b44-5debf5c2958f"). InnerVolumeSpecName "kube-api-access-jwbq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:27:52.440053 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.440020    2575 generic.go:356] "Generic (PLEG): container finished" podID="6f8e150f-4bf0-4b25-9b44-5debf5c2958f" containerID="9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172" exitCode=0
Jul 22 13:27:52.440215 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.440066    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv" event={"ID":"6f8e150f-4bf0-4b25-9b44-5debf5c2958f","Type":"ContainerDied","Data":"9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172"}
Jul 22 13:27:52.440215 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.440074    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"
Jul 22 13:27:52.440215 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.440130    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv" event={"ID":"6f8e150f-4bf0-4b25-9b44-5debf5c2958f","Type":"ContainerDied","Data":"4232ba95f37c5b23b934149c82b47bba82101a1744aabfbb698c654d58963d04"}
Jul 22 13:27:52.440215 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.440153    2575 scope.go:117] "RemoveContainer" containerID="9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172"
Jul 22 13:27:52.449136 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.449117    2575 scope.go:117] "RemoveContainer" containerID="9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172"
Jul 22 13:27:52.449393 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:27:52.449373    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172\": container with ID starting with 9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172 not found: ID does not exist" containerID="9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172"
Jul 22 13:27:52.449459 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.449406    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172"} err="failed to get container status \"9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172\": rpc error: code = NotFound desc = could not find container \"9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172\": container with ID starting with 9a8ecd1ec6ae7959ad868ec5d6ce37e845554b29769815aa836f9d48f1a49172 not found: ID does not exist"
Jul 22 13:27:52.463187 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.463162    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"]
Jul 22 13:27:52.464670 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.464650    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwbq6\" (UniqueName: \"kubernetes.io/projected/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-kube-api-access-jwbq6\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:27:52.464731 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.464673    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:27:52.464731 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.464684    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8e150f-4bf0-4b25-9b44-5debf5c2958f-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:27:52.471291 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.471269    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-16e9e526-59865b4447-cdqqv"]
Jul 22 13:27:52.948110 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:52.948076    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f8e150f-4bf0-4b25-9b44-5debf5c2958f" path="/var/lib/kubelet/pods/6f8e150f-4bf0-4b25-9b44-5debf5c2958f/volumes"
Jul 22 13:27:56.542972 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:56.542940    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"]
Jul 22 13:27:56.547269 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:56.547241    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-16e9e526-7hpw2"]
Jul 22 13:27:56.947507 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:27:56.947475    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f" path="/var/lib/kubelet/pods/32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f/volumes"
Jul 22 13:28:03.405818 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:03.405778    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"]
Jul 22 13:28:03.406267 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:03.406064    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy" containerID="cri-o://73fc3b73d6153a8f02470dff3aa36a5f179d54207f8eb47ce00b668cf5bc7d04" gracePeriod=30
Jul 22 13:28:08.531462 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.531430    2575 generic.go:356] "Generic (PLEG): container finished" podID="d22c0cc0-1339-435e-8979-4285cde324aa" containerID="73fc3b73d6153a8f02470dff3aa36a5f179d54207f8eb47ce00b668cf5bc7d04" exitCode=0
Jul 22 13:28:08.531839 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.531498    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" event={"ID":"d22c0cc0-1339-435e-8979-4285cde324aa","Type":"ContainerDied","Data":"73fc3b73d6153a8f02470dff3aa36a5f179d54207f8eb47ce00b668cf5bc7d04"}
Jul 22 13:28:08.531839 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.531561    2575 scope.go:117] "RemoveContainer" containerID="4486a9b0c3bf162f7d3caf0d86bc6315a6bd65a10fb1e1d124b6f70a9424dc63"
Jul 22 13:28:08.666128 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.666101    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:28:08.707527 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707439    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-istio-token\") pod \"d22c0cc0-1339-435e-8979-4285cde324aa\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") "
Jul 22 13:28:08.707527 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707489    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d22c0cc0-1339-435e-8979-4285cde324aa-istio-podinfo\") pod \"d22c0cc0-1339-435e-8979-4285cde324aa\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") "
Jul 22 13:28:08.707749 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707530    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-certs\") pod \"d22c0cc0-1339-435e-8979-4285cde324aa\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") "
Jul 22 13:28:08.707749 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707553    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-data\") pod \"d22c0cc0-1339-435e-8979-4285cde324aa\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") "
Jul 22 13:28:08.707749 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707584    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-socket\") pod \"d22c0cc0-1339-435e-8979-4285cde324aa\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") "
Jul 22 13:28:08.707749 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707612    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-envoy\") pod \"d22c0cc0-1339-435e-8979-4285cde324aa\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") "
Jul 22 13:28:08.707749 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707636    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d22c0cc0-1339-435e-8979-4285cde324aa-istiod-ca-cert\") pod \"d22c0cc0-1339-435e-8979-4285cde324aa\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") "
Jul 22 13:28:08.707749 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707686    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxdqg\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-kube-api-access-fxdqg\") pod \"d22c0cc0-1339-435e-8979-4285cde324aa\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") "
Jul 22 13:28:08.708087 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707756    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-credential-socket\") pod \"d22c0cc0-1339-435e-8979-4285cde324aa\" (UID: \"d22c0cc0-1339-435e-8979-4285cde324aa\") "
Jul 22 13:28:08.708087 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.707881    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d22c0cc0-1339-435e-8979-4285cde324aa" (UID: "d22c0cc0-1339-435e-8979-4285cde324aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:28:08.708087 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.708038    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d22c0cc0-1339-435e-8979-4285cde324aa" (UID: "d22c0cc0-1339-435e-8979-4285cde324aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:28:08.708272 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.708140    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d22c0cc0-1339-435e-8979-4285cde324aa" (UID: "d22c0cc0-1339-435e-8979-4285cde324aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:28:08.708272 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.708171    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d22c0cc0-1339-435e-8979-4285cde324aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d22c0cc0-1339-435e-8979-4285cde324aa" (UID: "d22c0cc0-1339-435e-8979-4285cde324aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:28:08.708603 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.708573    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "d22c0cc0-1339-435e-8979-4285cde324aa" (UID: "d22c0cc0-1339-435e-8979-4285cde324aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:28:08.708802 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.708646    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-credential-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:28:08.708802 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.708671    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:28:08.708802 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.708703    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-workload-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:28:08.708802 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.708719    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d22c0cc0-1339-435e-8979-4285cde324aa-istiod-ca-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:28:08.710067 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.710041    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "d22c0cc0-1339-435e-8979-4285cde324aa" (UID: "d22c0cc0-1339-435e-8979-4285cde324aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:28:08.710067 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.710056    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d22c0cc0-1339-435e-8979-4285cde324aa" (UID: "d22c0cc0-1339-435e-8979-4285cde324aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:28:08.710203 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.710156    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d22c0cc0-1339-435e-8979-4285cde324aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d22c0cc0-1339-435e-8979-4285cde324aa" (UID: "d22c0cc0-1339-435e-8979-4285cde324aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:28:08.710368 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.710344    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-kube-api-access-fxdqg" (OuterVolumeSpecName: "kube-api-access-fxdqg") pod "d22c0cc0-1339-435e-8979-4285cde324aa" (UID: "d22c0cc0-1339-435e-8979-4285cde324aa"). InnerVolumeSpecName "kube-api-access-fxdqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:28:08.809404 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.809369    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-data\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:28:08.809404 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.809392    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d22c0cc0-1339-435e-8979-4285cde324aa-istio-envoy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:28:08.809404 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.809402    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxdqg\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-kube-api-access-fxdqg\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:28:08.809404 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.809411    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d22c0cc0-1339-435e-8979-4285cde324aa-istio-token\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:28:08.809663 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:08.809419    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d22c0cc0-1339-435e-8979-4285cde324aa-istio-podinfo\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:28:09.537652 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:09.537617    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq" event={"ID":"d22c0cc0-1339-435e-8979-4285cde324aa","Type":"ContainerDied","Data":"adfd1f981093f8bfa5dcaf397ab2c84c1e19f72605684ec8b7e0343abd595d77"}
Jul 22 13:28:09.538175 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:09.537672    2575 scope.go:117] "RemoveContainer" containerID="73fc3b73d6153a8f02470dff3aa36a5f179d54207f8eb47ce00b668cf5bc7d04"
Jul 22 13:28:09.538175 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:09.537631    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"
Jul 22 13:28:09.558621 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:09.558591    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"]
Jul 22 13:28:09.565516 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:09.565481    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-16e9e526-openshift-default-58db46887-vrmfq"]
Jul 22 13:28:10.948483 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:10.948448    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" path="/var/lib/kubelet/pods/d22c0cc0-1339-435e-8979-4285cde324aa/volumes"
Jul 22 13:28:24.231683 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.231642    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"]
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232067    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy"
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232079    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy"
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232094    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f" containerName="revoke-keys"
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232099    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f" containerName="revoke-keys"
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232116    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f8e150f-4bf0-4b25-9b44-5debf5c2958f" containerName="maas-api"
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232121    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f8e150f-4bf0-4b25-9b44-5debf5c2958f" containerName="maas-api"
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232175    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy"
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232182    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="32a4d2a4-9b6e-4b7d-b13b-0d4be8ca576f" containerName="revoke-keys"
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232188    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy"
Jul 22 13:28:24.232276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.232194    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f8e150f-4bf0-4b25-9b44-5debf5c2958f" containerName="maas-api"
Jul 22 13:28:24.235353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.235337    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.238221 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.238198    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2b3e367f-openshift-default-dockercfg-wlpcz\""
Jul 22 13:28:24.238373 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.238206    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 13:28:24.252654 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.252629    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"]
Jul 22 13:28:24.342661 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.342634    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-token\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.342854 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.342667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-socket\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.342854 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.342686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-envoy\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.342854 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.342798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-certs\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.342854 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.342834    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-podinfo\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.343033 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.342879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-credential-socket\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.343033 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.342894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-data\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.343033 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.342909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq7vk\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-kube-api-access-mq7vk\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.343033 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.342986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c2840e-d84f-4e2f-aac4-54c137ce2657-istiod-ca-cert\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444250 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444210    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-certs\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444250 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444250    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-podinfo\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444539 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444292    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-credential-socket\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444539 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-data\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444539 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444339    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq7vk\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-kube-api-access-mq7vk\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444700 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444526    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c2840e-d84f-4e2f-aac4-54c137ce2657-istiod-ca-cert\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444700 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444607    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-token\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444700 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444639    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-socket\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444700 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444662    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-envoy\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444903 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444714    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-certs\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.444903 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444801    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-data\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.445052 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-credential-socket\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.445052 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.444970    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-socket\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.445283 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.445265    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c2840e-d84f-4e2f-aac4-54c137ce2657-istiod-ca-cert\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.446688 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.446670    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-envoy\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.446781 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.446750    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-podinfo\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.454620 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.454588    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-token\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.454744 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.454724    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq7vk\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-kube-api-access-mq7vk\") pod \"e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") " pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.550348 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.550262    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:24.697181 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.697156    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"]
Jul 22 13:28:24.699720 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:28:24.699693    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c2840e_d84f_4e2f_aac4_54c137ce2657.slice/crio-77cb570f293f9c4e09e4a4a2559dfd0d9475d41967ff64db0ce76a3e99b18b75 WatchSource:0}: Error finding container 77cb570f293f9c4e09e4a4a2559dfd0d9475d41967ff64db0ce76a3e99b18b75: Status 404 returned error can't find the container with id 77cb570f293f9c4e09e4a4a2559dfd0d9475d41967ff64db0ce76a3e99b18b75
Jul 22 13:28:24.702043 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.702003    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:28:24.702134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.702092    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:28:24.702173 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:24.702131    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:28:25.616879 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:25.616845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" event={"ID":"63c2840e-d84f-4e2f-aac4-54c137ce2657","Type":"ContainerStarted","Data":"16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352"}
Jul 22 13:28:25.617354 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:25.616899    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" event={"ID":"63c2840e-d84f-4e2f-aac4-54c137ce2657","Type":"ContainerStarted","Data":"77cb570f293f9c4e09e4a4a2559dfd0d9475d41967ff64db0ce76a3e99b18b75"}
Jul 22 13:28:25.641506 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:25.641456    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podStartSLOduration=1.641441164 podStartE2EDuration="1.641441164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:28:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:28:25.640260353 +0000 UTC m=+2057.932241397" watchObservedRunningTime="2026-07-22 13:28:25.641441164 +0000 UTC m=+2057.933422194"
Jul 22 13:28:26.551106 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:26.551068    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:26.555775 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:26.555752    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:26.622001 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:26.621964    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:26.622997 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:26.622978    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:27.715394 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.715358    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"]
Jul 22 13:28:27.715784 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.715706    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy"
Jul 22 13:28:27.715784 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.715717    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d22c0cc0-1339-435e-8979-4285cde324aa" containerName="istio-proxy"
Jul 22 13:28:27.718908 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.718891    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:27.722077 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.722053    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2b3e367f-serving-cert\""
Jul 22 13:28:27.722197 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.722187    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 13:28:27.722310 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.722293    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 13:28:27.723282 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.723261    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5td92\""
Jul 22 13:28:27.730151 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.730132    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"]
Jul 22 13:28:27.779229 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.779197    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkq6x\" (UniqueName: \"kubernetes.io/projected/18581fc8-eb08-487e-a983-fd2327b0be7d-kube-api-access-tkq6x\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:27.779229 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.779244    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18581fc8-eb08-487e-a983-fd2327b0be7d-maas-api-tls\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:27.779498 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.779280    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18581fc8-eb08-487e-a983-fd2327b0be7d-openshift-service-ca\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:27.880556 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.880526    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18581fc8-eb08-487e-a983-fd2327b0be7d-openshift-service-ca\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:27.880770 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.880592    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkq6x\" (UniqueName: \"kubernetes.io/projected/18581fc8-eb08-487e-a983-fd2327b0be7d-kube-api-access-tkq6x\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:27.880770 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.880629    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18581fc8-eb08-487e-a983-fd2327b0be7d-maas-api-tls\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:27.880770 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:28:27.880750    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-serving-cert: secret "maas-api-e2e-mt-2b3e367f-serving-cert" not found
Jul 22 13:28:27.880964 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:28:27.880805    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18581fc8-eb08-487e-a983-fd2327b0be7d-maas-api-tls podName:18581fc8-eb08-487e-a983-fd2327b0be7d nodeName:}" failed. No retries permitted until 2026-07-22 13:28:28.380787476 +0000 UTC m=+2060.672768487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/18581fc8-eb08-487e-a983-fd2327b0be7d-maas-api-tls") pod "maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5" (UID: "18581fc8-eb08-487e-a983-fd2327b0be7d") : secret "maas-api-e2e-mt-2b3e367f-serving-cert" not found
Jul 22 13:28:27.881229 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.881209    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18581fc8-eb08-487e-a983-fd2327b0be7d-openshift-service-ca\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:27.894214 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:27.894188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkq6x\" (UniqueName: \"kubernetes.io/projected/18581fc8-eb08-487e-a983-fd2327b0be7d-kube-api-access-tkq6x\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:28.386296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:28.386259    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18581fc8-eb08-487e-a983-fd2327b0be7d-maas-api-tls\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:28.388748 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:28.388724    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18581fc8-eb08-487e-a983-fd2327b0be7d-maas-api-tls\") pod \"maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:28.630265 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:28.630226    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:28.768254 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:28.768230    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"]
Jul 22 13:28:28.770307 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:28:28.770278    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18581fc8_eb08_487e_a983_fd2327b0be7d.slice/crio-d65f94a01c8e3ab0f95db5e8599dcf268a5c69c4bb000c15bafe7f35c7c1a873 WatchSource:0}: Error finding container d65f94a01c8e3ab0f95db5e8599dcf268a5c69c4bb000c15bafe7f35c7c1a873: Status 404 returned error can't find the container with id d65f94a01c8e3ab0f95db5e8599dcf268a5c69c4bb000c15bafe7f35c7c1a873
Jul 22 13:28:28.771799 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:28.771774    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 13:28:29.637921 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:29.637883    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5" event={"ID":"18581fc8-eb08-487e-a983-fd2327b0be7d","Type":"ContainerStarted","Data":"c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4"}
Jul 22 13:28:29.638159 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:29.638004    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5" event={"ID":"18581fc8-eb08-487e-a983-fd2327b0be7d","Type":"ContainerStarted","Data":"d65f94a01c8e3ab0f95db5e8599dcf268a5c69c4bb000c15bafe7f35c7c1a873"}
Jul 22 13:28:29.638159 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:29.638038    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:29.659462 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:29.659407    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5" podStartSLOduration=2.339398955 podStartE2EDuration="2.659390137s" totalImagesPullingTime="319.991182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:28:29.657529328 +0000 UTC m=+2061.949510363" watchObservedRunningTime="2026-07-22 13:28:29.659390137 +0000 UTC m=+2061.951371199"
Jul 22 13:28:35.651033 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:35.651001    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:28:41.776100 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:41.776000    2575 patch_prober.go:28] interesting pod/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": read tcp 10.133.0.2:40192->10.133.0.51:15021: read: connection reset by peer" start-of-body=
Jul 22 13:28:41.776100 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:41.776067    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": read tcp 10.133.0.2:40192->10.133.0.51:15021: read: connection reset by peer"
Jul 22 13:28:42.700372 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:42.700331    2575 generic.go:356] "Generic (PLEG): container finished" podID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerID="16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352" exitCode=0
Jul 22 13:28:42.700571 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:42.700432    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" event={"ID":"63c2840e-d84f-4e2f-aac4-54c137ce2657","Type":"ContainerDied","Data":"16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352"}
Jul 22 13:28:42.700868 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:42.700854    2575 scope.go:117] "RemoveContainer" containerID="16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352"
Jul 22 13:28:42.701868 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:42.701841    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:28:42.701989 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:42.701920    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:28:42.702063 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:42.701994    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:28:43.708288 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:43.707389    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" event={"ID":"63c2840e-d84f-4e2f-aac4-54c137ce2657","Type":"ContainerStarted","Data":"bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95"}
Jul 22 13:28:44.550709 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:44.550674    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:44.550890 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:44.550870    2575 patch_prober.go:28] interesting pod/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:28:44.550958 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:44.550912    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 13:28:45.550746 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:45.550711    2575 patch_prober.go:28] interesting pod/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:28:45.551193 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:45.550765    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 13:28:46.551554 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:46.551514    2575 patch_prober.go:28] interesting pod/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:28:46.551986 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:46.551575    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 13:28:47.551217 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:47.551177    2575 patch_prober.go:28] interesting pod/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:28:47.551542 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:47.551504    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 13:28:48.551135 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:48.551098    2575 patch_prober.go:28] interesting pod/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:28:48.551529 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:48.551172    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 13:28:49.551355 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:49.551320    2575 patch_prober.go:28] interesting pod/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:28:49.551743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:49.551391    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 13:28:50.554511 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:50.554481    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:50.554885 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:50.554715    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:28:50.555443 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:28:50.555425    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:29:05.642501 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:05.642462    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"]
Jul 22 13:29:05.645319 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:05.645297    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"
Jul 22 13:29:05.648099 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:05.648078    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:29:05.659535 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:05.659500    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"]
Jul 22 13:29:05.730348 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:05.730310    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008f2b00-528d-4901-a2cb-a0681a9ad536-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f\" (UID: \"008f2b00-528d-4901-a2cb-a0681a9ad536\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"
Jul 22 13:29:05.830832 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:05.830801    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008f2b00-528d-4901-a2cb-a0681a9ad536-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f\" (UID: \"008f2b00-528d-4901-a2cb-a0681a9ad536\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"
Jul 22 13:29:05.831465 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:05.831444    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008f2b00-528d-4901-a2cb-a0681a9ad536-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f\" (UID: \"008f2b00-528d-4901-a2cb-a0681a9ad536\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"
Jul 22 13:29:05.957719 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:05.957626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"
Jul 22 13:29:06.085245 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:06.085216    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"]
Jul 22 13:29:06.087398 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:29:06.087374    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod008f2b00_528d_4901_a2cb_a0681a9ad536.slice/crio-6f97a9c4088e69f684776fec4defc055eeac7a4f6d24a179a71696a8b0a552b5 WatchSource:0}: Error finding container 6f97a9c4088e69f684776fec4defc055eeac7a4f6d24a179a71696a8b0a552b5: Status 404 returned error can't find the container with id 6f97a9c4088e69f684776fec4defc055eeac7a4f6d24a179a71696a8b0a552b5
Jul 22 13:29:06.826103 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:06.826068    2575 generic.go:356] "Generic (PLEG): container finished" podID="008f2b00-528d-4901-a2cb-a0681a9ad536" containerID="583d7b91510025ee1336b1cbfbef26e506fe7b54b47e29287a8dde6cef18ee70" exitCode=0
Jul 22 13:29:06.826475 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:06.826112    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f" event={"ID":"008f2b00-528d-4901-a2cb-a0681a9ad536","Type":"ContainerDied","Data":"583d7b91510025ee1336b1cbfbef26e506fe7b54b47e29287a8dde6cef18ee70"}
Jul 22 13:29:06.826475 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:06.826169    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f" event={"ID":"008f2b00-528d-4901-a2cb-a0681a9ad536","Type":"ContainerStarted","Data":"6f97a9c4088e69f684776fec4defc055eeac7a4f6d24a179a71696a8b0a552b5"}
Jul 22 13:29:07.959614 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:07.959588    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"
Jul 22 13:29:08.052208 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:08.052176    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008f2b00-528d-4901-a2cb-a0681a9ad536-maas-api-service-ca\") pod \"008f2b00-528d-4901-a2cb-a0681a9ad536\" (UID: \"008f2b00-528d-4901-a2cb-a0681a9ad536\") "
Jul 22 13:29:08.052530 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:08.052511    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/008f2b00-528d-4901-a2cb-a0681a9ad536-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "008f2b00-528d-4901-a2cb-a0681a9ad536" (UID: "008f2b00-528d-4901-a2cb-a0681a9ad536"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:29:08.153743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:08.153665    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008f2b00-528d-4901-a2cb-a0681a9ad536-maas-api-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:29:08.837547 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:08.837519    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"
Jul 22 13:29:08.837737 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:08.837556    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f" event={"ID":"008f2b00-528d-4901-a2cb-a0681a9ad536","Type":"ContainerDied","Data":"6f97a9c4088e69f684776fec4defc055eeac7a4f6d24a179a71696a8b0a552b5"}
Jul 22 13:29:08.837737 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:08.837611    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f97a9c4088e69f684776fec4defc055eeac7a4f6d24a179a71696a8b0a552b5"
Jul 22 13:29:08.966152 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:08.966127    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:29:08.971720 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:08.971696    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:29:21.076844 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.076808    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"]
Jul 22 13:29:21.077434 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.077085    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5" podUID="18581fc8-eb08-487e-a983-fd2327b0be7d" containerName="maas-api" containerID="cri-o://c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4" gracePeriod=30
Jul 22 13:29:21.320650 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.320626    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:29:21.470573 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.470482    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18581fc8-eb08-487e-a983-fd2327b0be7d-maas-api-tls\") pod \"18581fc8-eb08-487e-a983-fd2327b0be7d\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") "
Jul 22 13:29:21.470731 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.470617    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkq6x\" (UniqueName: \"kubernetes.io/projected/18581fc8-eb08-487e-a983-fd2327b0be7d-kube-api-access-tkq6x\") pod \"18581fc8-eb08-487e-a983-fd2327b0be7d\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") "
Jul 22 13:29:21.470731 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.470668    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18581fc8-eb08-487e-a983-fd2327b0be7d-openshift-service-ca\") pod \"18581fc8-eb08-487e-a983-fd2327b0be7d\" (UID: \"18581fc8-eb08-487e-a983-fd2327b0be7d\") "
Jul 22 13:29:21.471052 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.471030    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18581fc8-eb08-487e-a983-fd2327b0be7d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18581fc8-eb08-487e-a983-fd2327b0be7d" (UID: "18581fc8-eb08-487e-a983-fd2327b0be7d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:29:21.472717 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.472694    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18581fc8-eb08-487e-a983-fd2327b0be7d-kube-api-access-tkq6x" (OuterVolumeSpecName: "kube-api-access-tkq6x") pod "18581fc8-eb08-487e-a983-fd2327b0be7d" (UID: "18581fc8-eb08-487e-a983-fd2327b0be7d"). InnerVolumeSpecName "kube-api-access-tkq6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:29:21.472820 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.472749    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18581fc8-eb08-487e-a983-fd2327b0be7d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18581fc8-eb08-487e-a983-fd2327b0be7d" (UID: "18581fc8-eb08-487e-a983-fd2327b0be7d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:29:21.572264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.572230    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkq6x\" (UniqueName: \"kubernetes.io/projected/18581fc8-eb08-487e-a983-fd2327b0be7d-kube-api-access-tkq6x\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:29:21.572264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.572259    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18581fc8-eb08-487e-a983-fd2327b0be7d-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:29:21.572264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.572269    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18581fc8-eb08-487e-a983-fd2327b0be7d-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:29:21.900398 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.900354    2575 generic.go:356] "Generic (PLEG): container finished" podID="18581fc8-eb08-487e-a983-fd2327b0be7d" containerID="c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4" exitCode=0
Jul 22 13:29:21.900632 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.900430    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"
Jul 22 13:29:21.900632 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.900467    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5" event={"ID":"18581fc8-eb08-487e-a983-fd2327b0be7d","Type":"ContainerDied","Data":"c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4"}
Jul 22 13:29:21.900632 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.900525    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5" event={"ID":"18581fc8-eb08-487e-a983-fd2327b0be7d","Type":"ContainerDied","Data":"d65f94a01c8e3ab0f95db5e8599dcf268a5c69c4bb000c15bafe7f35c7c1a873"}
Jul 22 13:29:21.900632 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.900552    2575 scope.go:117] "RemoveContainer" containerID="c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4"
Jul 22 13:29:21.909355 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.909337    2575 scope.go:117] "RemoveContainer" containerID="c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4"
Jul 22 13:29:21.909626 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:29:21.909595    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4\": container with ID starting with c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4 not found: ID does not exist" containerID="c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4"
Jul 22 13:29:21.909716 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.909636    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4"} err="failed to get container status \"c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4\": rpc error: code = NotFound desc = could not find container \"c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4\": container with ID starting with c2598f0874187275aea871d1c9eed8e478fd6390dc05bdc03ce6352b609f7df4 not found: ID does not exist"
Jul 22 13:29:21.923008 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.922982    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"]
Jul 22 13:29:21.927258 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:21.927236    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2b3e367f-5d46fb747b-pljt5"]
Jul 22 13:29:22.952912 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:22.952877    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18581fc8-eb08-487e-a983-fd2327b0be7d" path="/var/lib/kubelet/pods/18581fc8-eb08-487e-a983-fd2327b0be7d/volumes"
Jul 22 13:29:25.727813 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:25.727778    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"]
Jul 22 13:29:25.733105 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:25.733075    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b3e367f-drd4f"]
Jul 22 13:29:26.947599 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:29:26.947565    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="008f2b00-528d-4901-a2cb-a0681a9ad536" path="/var/lib/kubelet/pods/008f2b00-528d-4901-a2cb-a0681a9ad536/volumes"
Jul 22 13:30:00.138701 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.138618    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb"]
Jul 22 13:30:00.139151 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.139012    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18581fc8-eb08-487e-a983-fd2327b0be7d" containerName="maas-api"
Jul 22 13:30:00.139151 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.139027    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="18581fc8-eb08-487e-a983-fd2327b0be7d" containerName="maas-api"
Jul 22 13:30:00.139151 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.139042    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="008f2b00-528d-4901-a2cb-a0681a9ad536" containerName="revoke-keys"
Jul 22 13:30:00.139151 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.139047    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="008f2b00-528d-4901-a2cb-a0681a9ad536" containerName="revoke-keys"
Jul 22 13:30:00.139151 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.139116    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="008f2b00-528d-4901-a2cb-a0681a9ad536" containerName="revoke-keys"
Jul 22 13:30:00.139151 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.139129    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="18581fc8-eb08-487e-a983-fd2327b0be7d" containerName="maas-api"
Jul 22 13:30:00.142006 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.141990    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb"
Jul 22 13:30:00.144330 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.144314    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:30:00.150331 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.150310    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb"]
Jul 22 13:30:00.153299 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.153279    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb"
Jul 22 13:30:00.288616 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:00.288575    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb"]
Jul 22 13:30:00.290946 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:30:00.290891    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea2e02e_00f7_425b_ac08_c188194cde01.slice/crio-81a7bacc7a52f4e5eed3c80d5e7753479a033e138bdbba71629faa8c7238d59b WatchSource:0}: Error finding container 81a7bacc7a52f4e5eed3c80d5e7753479a033e138bdbba71629faa8c7238d59b: Status 404 returned error can't find the container with id 81a7bacc7a52f4e5eed3c80d5e7753479a033e138bdbba71629faa8c7238d59b
Jul 22 13:30:01.107923 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:01.107887    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb" event={"ID":"4ea2e02e-00f7-425b-ac08-c188194cde01","Type":"ContainerStarted","Data":"49d7c22f88027979152104cce341f7d1ed6ec08b572420d00a81f0527c534446"}
Jul 22 13:30:01.108118 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:01.107982    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb" event={"ID":"4ea2e02e-00f7-425b-ac08-c188194cde01","Type":"ContainerStarted","Data":"81a7bacc7a52f4e5eed3c80d5e7753479a033e138bdbba71629faa8c7238d59b"}
Jul 22 13:30:01.123920 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:01.123870    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb" podStartSLOduration=1.123854694 podStartE2EDuration="1.123854694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:30:01.122724999 +0000 UTC m=+2153.414706024" watchObservedRunningTime="2026-07-22 13:30:01.123854694 +0000 UTC m=+2153.415835727"
Jul 22 13:30:02.117924 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:02.117893    2575 generic.go:356] "Generic (PLEG): container finished" podID="4ea2e02e-00f7-425b-ac08-c188194cde01" containerID="49d7c22f88027979152104cce341f7d1ed6ec08b572420d00a81f0527c534446" exitCode=0
Jul 22 13:30:02.118324 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:02.117969    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb" event={"ID":"4ea2e02e-00f7-425b-ac08-c188194cde01","Type":"ContainerDied","Data":"49d7c22f88027979152104cce341f7d1ed6ec08b572420d00a81f0527c534446"}
Jul 22 13:30:03.267288 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:03.267265    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb"
Jul 22 13:30:04.130067 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:04.130037    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb"
Jul 22 13:30:04.130251 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:04.130039    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745450-6gcqb" event={"ID":"4ea2e02e-00f7-425b-ac08-c188194cde01","Type":"ContainerDied","Data":"81a7bacc7a52f4e5eed3c80d5e7753479a033e138bdbba71629faa8c7238d59b"}
Jul 22 13:30:04.130251 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:04.130159    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81a7bacc7a52f4e5eed3c80d5e7753479a033e138bdbba71629faa8c7238d59b"
Jul 22 13:30:09.066498 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:09.066467    2575 scope.go:117] "RemoveContainer" containerID="0cf986a479533edec70dcb6fdef907001663e015a0a50d03dd1ec2e67f159761"
Jul 22 13:30:09.074911 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:09.074891    2575 scope.go:117] "RemoveContainer" containerID="3c027b2869381d68f20b7ae8030db7243efd9426ba68959f9e9c725e01e58a13"
Jul 22 13:30:15.834409 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:15.834370    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"]
Jul 22 13:30:15.834902 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:15.834582    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy" containerID="cri-o://bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95" gracePeriod=30
Jul 22 13:30:20.557056 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:20.557018    2575 patch_prober.go:28] interesting pod/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 13:30:20.557435 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:20.557089    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 13:30:21.104602 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.104523    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:30:21.120283 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120251    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-socket\") pod \"63c2840e-d84f-4e2f-aac4-54c137ce2657\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") "
Jul 22 13:30:21.120283 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120298    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-credential-socket\") pod \"63c2840e-d84f-4e2f-aac4-54c137ce2657\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") "
Jul 22 13:30:21.120546 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120327    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-data\") pod \"63c2840e-d84f-4e2f-aac4-54c137ce2657\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") "
Jul 22 13:30:21.120546 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120363    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-token\") pod \"63c2840e-d84f-4e2f-aac4-54c137ce2657\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") "
Jul 22 13:30:21.120546 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120401    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq7vk\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-kube-api-access-mq7vk\") pod \"63c2840e-d84f-4e2f-aac4-54c137ce2657\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") "
Jul 22 13:30:21.120546 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120480    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-envoy\") pod \"63c2840e-d84f-4e2f-aac4-54c137ce2657\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") "
Jul 22 13:30:21.120546 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120504    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-podinfo\") pod \"63c2840e-d84f-4e2f-aac4-54c137ce2657\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") "
Jul 22 13:30:21.120546 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120535    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c2840e-d84f-4e2f-aac4-54c137ce2657-istiod-ca-cert\") pod \"63c2840e-d84f-4e2f-aac4-54c137ce2657\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") "
Jul 22 13:30:21.120857 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120570    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-certs\") pod \"63c2840e-d84f-4e2f-aac4-54c137ce2657\" (UID: \"63c2840e-d84f-4e2f-aac4-54c137ce2657\") "
Jul 22 13:30:21.120857 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120577    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "63c2840e-d84f-4e2f-aac4-54c137ce2657" (UID: "63c2840e-d84f-4e2f-aac4-54c137ce2657"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:30:21.120998 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.120864    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:30:21.121134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.121113    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "63c2840e-d84f-4e2f-aac4-54c137ce2657" (UID: "63c2840e-d84f-4e2f-aac4-54c137ce2657"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:30:21.121434 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.121409    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63c2840e-d84f-4e2f-aac4-54c137ce2657-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "63c2840e-d84f-4e2f-aac4-54c137ce2657" (UID: "63c2840e-d84f-4e2f-aac4-54c137ce2657"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:30:21.121644 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.121622    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "63c2840e-d84f-4e2f-aac4-54c137ce2657" (UID: "63c2840e-d84f-4e2f-aac4-54c137ce2657"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:30:21.121780 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.121757    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-data" (OuterVolumeSpecName: "istio-data") pod "63c2840e-d84f-4e2f-aac4-54c137ce2657" (UID: "63c2840e-d84f-4e2f-aac4-54c137ce2657"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:30:21.123169 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.123134    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-kube-api-access-mq7vk" (OuterVolumeSpecName: "kube-api-access-mq7vk") pod "63c2840e-d84f-4e2f-aac4-54c137ce2657" (UID: "63c2840e-d84f-4e2f-aac4-54c137ce2657"). InnerVolumeSpecName "kube-api-access-mq7vk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:30:21.123410 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.123378    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-token" (OuterVolumeSpecName: "istio-token") pod "63c2840e-d84f-4e2f-aac4-54c137ce2657" (UID: "63c2840e-d84f-4e2f-aac4-54c137ce2657"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:30:21.123684 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.123632    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "63c2840e-d84f-4e2f-aac4-54c137ce2657" (UID: "63c2840e-d84f-4e2f-aac4-54c137ce2657"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:30:21.123772 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.123726    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "63c2840e-d84f-4e2f-aac4-54c137ce2657" (UID: "63c2840e-d84f-4e2f-aac4-54c137ce2657"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:30:21.219080 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.219038    2575 generic.go:356] "Generic (PLEG): container finished" podID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerID="bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95" exitCode=0
Jul 22 13:30:21.219080 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.219095    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"
Jul 22 13:30:21.219296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.219139    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" event={"ID":"63c2840e-d84f-4e2f-aac4-54c137ce2657","Type":"ContainerDied","Data":"bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95"}
Jul 22 13:30:21.219296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.219213    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87" event={"ID":"63c2840e-d84f-4e2f-aac4-54c137ce2657","Type":"ContainerDied","Data":"77cb570f293f9c4e09e4a4a2559dfd0d9475d41967ff64db0ce76a3e99b18b75"}
Jul 22 13:30:21.219296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.219284    2575 scope.go:117] "RemoveContainer" containerID="bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95"
Jul 22 13:30:21.222085 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.222059    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-credential-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:30:21.222210 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.222089    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-data\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:30:21.222210 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.222103    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-token\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:30:21.222210 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.222118    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq7vk\" (UniqueName: \"kubernetes.io/projected/63c2840e-d84f-4e2f-aac4-54c137ce2657-kube-api-access-mq7vk\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:30:21.222210 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.222131    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-envoy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:30:21.222210 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.222146    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c2840e-d84f-4e2f-aac4-54c137ce2657-istio-podinfo\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:30:21.222210 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.222159    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c2840e-d84f-4e2f-aac4-54c137ce2657-istiod-ca-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:30:21.222210 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.222173    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c2840e-d84f-4e2f-aac4-54c137ce2657-workload-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:30:21.230252 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.230226    2575 scope.go:117] "RemoveContainer" containerID="16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352"
Jul 22 13:30:21.241673 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.241654    2575 scope.go:117] "RemoveContainer" containerID="bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95"
Jul 22 13:30:21.242003 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:30:21.241983    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95\": container with ID starting with bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95 not found: ID does not exist" containerID="bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95"
Jul 22 13:30:21.242077 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.242013    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95"} err="failed to get container status \"bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95\": rpc error: code = NotFound desc = could not find container \"bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95\": container with ID starting with bb2905a1d04092e3f1260cf3ba2add758b5ad302238e12ce53f6973a0fe4bd95 not found: ID does not exist"
Jul 22 13:30:21.242077 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.242034    2575 scope.go:117] "RemoveContainer" containerID="16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352"
Jul 22 13:30:21.242324 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:30:21.242297    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352\": container with ID starting with 16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352 not found: ID does not exist" containerID="16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352"
Jul 22 13:30:21.242427 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.242327    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352"} err="failed to get container status \"16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352\": rpc error: code = NotFound desc = could not find container \"16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352\": container with ID starting with 16787fab9f4e2c6ce02110f1dfecc625f42f8482b68f387846bb7389bcbba352 not found: ID does not exist"
Jul 22 13:30:21.258823 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.258792    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"]
Jul 22 13:30:21.265426 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:21.265399    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2b3e367f-openshift-default-7b69698bd7-mqt87"]
Jul 22 13:30:22.948008 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:30:22.947976    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" path="/var/lib/kubelet/pods/63c2840e-d84f-4e2f-aac4-54c137ce2657/volumes"
Jul 22 13:31:09.100978 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:09.100919    2575 scope.go:117] "RemoveContainer" containerID="098cb41fcf10ce01c4501af93354d68a5960b353697920637131b93ff219d659"
Jul 22 13:31:15.228727 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.228698    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"]
Jul 22 13:31:15.229264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.229077    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea2e02e-00f7-425b-ac08-c188194cde01" containerName="cleanup"
Jul 22 13:31:15.229264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.229090    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea2e02e-00f7-425b-ac08-c188194cde01" containerName="cleanup"
Jul 22 13:31:15.229264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.229100    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy"
Jul 22 13:31:15.229264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.229109    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy"
Jul 22 13:31:15.229264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.229132    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy"
Jul 22 13:31:15.229264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.229138    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy"
Jul 22 13:31:15.229264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.229216    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea2e02e-00f7-425b-ac08-c188194cde01" containerName="cleanup"
Jul 22 13:31:15.229264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.229226    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy"
Jul 22 13:31:15.229264 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.229233    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="63c2840e-d84f-4e2f-aac4-54c137ce2657" containerName="istio-proxy"
Jul 22 13:31:15.232159 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.232138    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.234908 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.234884    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cc03a6-openshift-default-dockercfg-b2pjl\""
Jul 22 13:31:15.235164 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.235149    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 13:31:15.246641 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.246617    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"]
Jul 22 13:31:15.309540 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.309508    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8hdb\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-kube-api-access-k8hdb\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.309713 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.309557    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istiod-ca-cert\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.309713 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.309615    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-data\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.309713 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.309632    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-token\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.309713 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.309689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-podinfo\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.309713 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.309712    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-envoy\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.309889 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.309768    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-socket\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.309889 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.309791    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-credential-socket\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.309889 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.309806    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-certs\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411157 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411117    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-socket\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411157 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411159    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-credential-socket\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411428 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-certs\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411428 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411204    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8hdb\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-kube-api-access-k8hdb\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411428 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411232    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istiod-ca-cert\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411428 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411271    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-data\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411428 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411289    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-token\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411428 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411327    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-podinfo\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411428 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411352    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-envoy\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411788 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411586    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-certs\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411788 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411620    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-socket\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411788 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411679    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-credential-socket\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.411974 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.411899    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-data\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.412051 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.412001    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istiod-ca-cert\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.413853 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.413832    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-envoy\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.413979 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.413958    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-podinfo\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.422948 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.422908    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-token\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.423268 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.423252    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8hdb\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-kube-api-access-k8hdb\") pod \"e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") " pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.542941 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.542847    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:15.682814 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.682779    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"]
Jul 22 13:31:15.684264 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:31:15.684241    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf859252f_c9b6_44ef_8bbe_3d19fe3fdbd0.slice/crio-16e74c00f87e55c0db885b93424cec93c1fd2a03ca3eb77bd0a1bdc83adda626 WatchSource:0}: Error finding container 16e74c00f87e55c0db885b93424cec93c1fd2a03ca3eb77bd0a1bdc83adda626: Status 404 returned error can't find the container with id 16e74c00f87e55c0db885b93424cec93c1fd2a03ca3eb77bd0a1bdc83adda626
Jul 22 13:31:15.686356 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.686324    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:15.686444 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.686388    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:15.686444 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:15.686414    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:16.498729 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:16.498696    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" event={"ID":"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0","Type":"ContainerStarted","Data":"9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196"}
Jul 22 13:31:16.499164 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:16.498752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" event={"ID":"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0","Type":"ContainerStarted","Data":"16e74c00f87e55c0db885b93424cec93c1fd2a03ca3eb77bd0a1bdc83adda626"}
Jul 22 13:31:16.526399 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:16.526345    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" podStartSLOduration=1.526328526 podStartE2EDuration="1.526328526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:31:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:31:16.525034557 +0000 UTC m=+2228.817015614" watchObservedRunningTime="2026-07-22 13:31:16.526328526 +0000 UTC m=+2228.818309556"
Jul 22 13:31:16.543003 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:16.542972    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:16.547845 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:16.547818    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:17.504476 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:17.504437    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:17.505503 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:17.505483    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:18.657239 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.657207    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"]
Jul 22 13:31:18.663891 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.662218    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:18.666039 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.666012    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5td92\""
Jul 22 13:31:18.667640 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.667615    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 13:31:18.668052 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.668032    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-cc03a6-serving-cert\""
Jul 22 13:31:18.670553 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.670353    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 13:31:18.676790 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.676702    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"]
Jul 22 13:31:18.742846 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.742799    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d92d7f-1bc7-4023-bded-ea31baac12f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:18.743054 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.742881    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d92d7f-1bc7-4023-bded-ea31baac12f4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:18.743054 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.742913    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6vjz\" (UniqueName: \"kubernetes.io/projected/28d92d7f-1bc7-4023-bded-ea31baac12f4-kube-api-access-x6vjz\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:18.844046 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.844003    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d92d7f-1bc7-4023-bded-ea31baac12f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:18.844230 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.844079    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d92d7f-1bc7-4023-bded-ea31baac12f4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:18.844230 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.844108    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6vjz\" (UniqueName: \"kubernetes.io/projected/28d92d7f-1bc7-4023-bded-ea31baac12f4-kube-api-access-x6vjz\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:18.844230 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:31:18.844171    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-serving-cert: secret "maas-api-e2e-inf-a-cc03a6-serving-cert" not found
Jul 22 13:31:18.844334 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:31:18.844249    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28d92d7f-1bc7-4023-bded-ea31baac12f4-maas-api-tls podName:28d92d7f-1bc7-4023-bded-ea31baac12f4 nodeName:}" failed. No retries permitted until 2026-07-22 13:31:19.34422926 +0000 UTC m=+2231.636210276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28d92d7f-1bc7-4023-bded-ea31baac12f4-maas-api-tls") pod "maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm" (UID: "28d92d7f-1bc7-4023-bded-ea31baac12f4") : secret "maas-api-e2e-inf-a-cc03a6-serving-cert" not found
Jul 22 13:31:18.844951 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.844910    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d92d7f-1bc7-4023-bded-ea31baac12f4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:18.863791 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:18.863764    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6vjz\" (UniqueName: \"kubernetes.io/projected/28d92d7f-1bc7-4023-bded-ea31baac12f4-kube-api-access-x6vjz\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:19.348621 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:19.348583    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d92d7f-1bc7-4023-bded-ea31baac12f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:19.351252 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:19.351228    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d92d7f-1bc7-4023-bded-ea31baac12f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:19.578529 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:19.578502    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:19.752012 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:19.751982    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"]
Jul 22 13:31:19.754794 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:31:19.754766    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d92d7f_1bc7_4023_bded_ea31baac12f4.slice/crio-119bfcafc79a841ef881a894a5948fb790e55a35a2ab0bbfa613a5d6e3ce993a WatchSource:0}: Error finding container 119bfcafc79a841ef881a894a5948fb790e55a35a2ab0bbfa613a5d6e3ce993a: Status 404 returned error can't find the container with id 119bfcafc79a841ef881a894a5948fb790e55a35a2ab0bbfa613a5d6e3ce993a
Jul 22 13:31:20.521862 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:20.521764    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm" event={"ID":"28d92d7f-1bc7-4023-bded-ea31baac12f4","Type":"ContainerStarted","Data":"74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2"}
Jul 22 13:31:20.522044 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:20.521881    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:20.522044 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:20.521919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm" event={"ID":"28d92d7f-1bc7-4023-bded-ea31baac12f4","Type":"ContainerStarted","Data":"119bfcafc79a841ef881a894a5948fb790e55a35a2ab0bbfa613a5d6e3ce993a"}
Jul 22 13:31:20.549328 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:20.549275    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm" podStartSLOduration=2.077040786 podStartE2EDuration="2.549260249s" totalImagesPullingTime="472.219463ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:31:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:31:20.548531463 +0000 UTC m=+2232.840512508" watchObservedRunningTime="2026-07-22 13:31:20.549260249 +0000 UTC m=+2232.841241281"
Jul 22 13:31:23.753084 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.753034    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"]
Jul 22 13:31:23.756743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.756719    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.759803 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.759779    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-aa3ea2-openshift-default-dockercfg-hdrt2\""
Jul 22 13:31:23.779965 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.779915    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"]
Jul 22 13:31:23.894187 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.894139    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-credential-socket\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.894383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.894201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-podinfo\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.894383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.894282    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-certs\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.894383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.894344    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-envoy\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.894383 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.894360    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-data\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.894548 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.894386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-token\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.894548 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.894416    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-socket\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.894548 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.894486    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istiod-ca-cert\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.894548 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.894516    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr8mk\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-kube-api-access-dr8mk\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.995451 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-socket\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.995631 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995519    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istiod-ca-cert\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.995631 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995576    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr8mk\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-kube-api-access-dr8mk\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.995803 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995645    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-credential-socket\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.995803 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995700    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-podinfo\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.995803 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995728    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-certs\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.995803 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995792    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-envoy\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.996051 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995815    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-data\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.996051 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-token\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.996051 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.995880    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-socket\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.996185 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.996132    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-certs\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.996247 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.996214    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-data\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.996296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.996287    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istiod-ca-cert\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.996347 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.996323    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-credential-socket\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.998103 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.998074    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-envoy\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:23.998385 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:23.998356    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-podinfo\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:24.009186 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.009120    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-token\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:24.009292 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.009212    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr8mk\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-kube-api-access-dr8mk\") pod \"e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") " pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:24.067349 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.067313    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:24.220586 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.220559    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"]
Jul 22 13:31:24.222203 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:31:24.222172    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd2d9c9_52c5_44d7_8a90_f5c13ebf24b3.slice/crio-6f92507878b9f1b801d90fd5ee0f51b64254298689fd88dfbaa99013d1dca861 WatchSource:0}: Error finding container 6f92507878b9f1b801d90fd5ee0f51b64254298689fd88dfbaa99013d1dca861: Status 404 returned error can't find the container with id 6f92507878b9f1b801d90fd5ee0f51b64254298689fd88dfbaa99013d1dca861
Jul 22 13:31:24.224439 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.224407    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:24.224500 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.224472    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:24.224544 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.224500    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:24.544418 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.544295    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" event={"ID":"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3","Type":"ContainerStarted","Data":"aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701"}
Jul 22 13:31:24.544418 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.544369    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" event={"ID":"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3","Type":"ContainerStarted","Data":"6f92507878b9f1b801d90fd5ee0f51b64254298689fd88dfbaa99013d1dca861"}
Jul 22 13:31:24.576125 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:24.576071    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" podStartSLOduration=1.57605388 podStartE2EDuration="1.57605388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:31:24.575380828 +0000 UTC m=+2236.867361864" watchObservedRunningTime="2026-07-22 13:31:24.57605388 +0000 UTC m=+2236.868034910"
Jul 22 13:31:25.067995 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:25.067957    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:25.073020 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:25.072982    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:25.549202 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:25.549166    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:25.550185 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:25.550166    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:26.533343 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.533306    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:31:26.747762 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.747725    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"]
Jul 22 13:31:26.752098 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.752074    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:26.754733 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.754708    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-aa3ea2-serving-cert\""
Jul 22 13:31:26.762586 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.762560    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"]
Jul 22 13:31:26.821153 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.821122    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bad330d-917e-4a99-a40c-0aee46e4d505-maas-api-tls\") pod \"maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:26.821341 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.821167    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc48g\" (UniqueName: \"kubernetes.io/projected/7bad330d-917e-4a99-a40c-0aee46e4d505-kube-api-access-cc48g\") pod \"maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:26.821341 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.821259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bad330d-917e-4a99-a40c-0aee46e4d505-openshift-service-ca\") pod \"maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:26.922913 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.922870    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bad330d-917e-4a99-a40c-0aee46e4d505-openshift-service-ca\") pod \"maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:26.923159 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.923037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bad330d-917e-4a99-a40c-0aee46e4d505-maas-api-tls\") pod \"maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:26.923159 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.923065    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc48g\" (UniqueName: \"kubernetes.io/projected/7bad330d-917e-4a99-a40c-0aee46e4d505-kube-api-access-cc48g\") pod \"maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:26.923586 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.923562    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bad330d-917e-4a99-a40c-0aee46e4d505-openshift-service-ca\") pod \"maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:26.925506 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.925484    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bad330d-917e-4a99-a40c-0aee46e4d505-maas-api-tls\") pod \"maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:26.933032 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:26.933006    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc48g\" (UniqueName: \"kubernetes.io/projected/7bad330d-917e-4a99-a40c-0aee46e4d505-kube-api-access-cc48g\") pod \"maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:27.065481 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:27.065427    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:27.214441 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:27.214416    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"]
Jul 22 13:31:27.216514 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:31:27.216484    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bad330d_917e_4a99_a40c_0aee46e4d505.slice/crio-db2404b3ba76f54d2fd0990922e05eb28e9ec2942ff5784fbdd54eb2aecb5c6f WatchSource:0}: Error finding container db2404b3ba76f54d2fd0990922e05eb28e9ec2942ff5784fbdd54eb2aecb5c6f: Status 404 returned error can't find the container with id db2404b3ba76f54d2fd0990922e05eb28e9ec2942ff5784fbdd54eb2aecb5c6f
Jul 22 13:31:27.560332 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:27.560242    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb" event={"ID":"7bad330d-917e-4a99-a40c-0aee46e4d505","Type":"ContainerStarted","Data":"db2404b3ba76f54d2fd0990922e05eb28e9ec2942ff5784fbdd54eb2aecb5c6f"}
Jul 22 13:31:28.567346 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:28.567307    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb" event={"ID":"7bad330d-917e-4a99-a40c-0aee46e4d505","Type":"ContainerStarted","Data":"b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97"}
Jul 22 13:31:28.567860 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:28.567405    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:28.586074 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:28.585998    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb" podStartSLOduration=2.117043617 podStartE2EDuration="2.585982024s" totalImagesPullingTime="468.938407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:31:28.585078753 +0000 UTC m=+2240.877059787" watchObservedRunningTime="2026-07-22 13:31:28.585982024 +0000 UTC m=+2240.877963089"
Jul 22 13:31:33.546168 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:33.546125    2575 patch_prober.go:28] interesting pod/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 22 13:31:33.546676 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:33.546196    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 22 13:31:34.577594 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:34.577567    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:31:35.614459 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:35.614420    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" event={"ID":"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0","Type":"ContainerDied","Data":"9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196"}
Jul 22 13:31:35.615005 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:35.614405    2575 generic.go:356] "Generic (PLEG): container finished" podID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerID="9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196" exitCode=0
Jul 22 13:31:35.615005 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:35.614999    2575 scope.go:117] "RemoveContainer" containerID="9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196"
Jul 22 13:31:35.616082 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:35.616054    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:35.616196 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:35.616155    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:35.616262 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:35.616200    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:36.621952 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:36.621912    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" event={"ID":"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0","Type":"ContainerStarted","Data":"7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a"}
Jul 22 13:31:37.543319 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:37.543213    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:37.543694 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:37.543660    2575 patch_prober.go:28] interesting pod/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 22 13:31:37.543813 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:37.543721    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 22 13:31:38.543986 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:38.543440    2575 patch_prober.go:28] interesting pod/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 22 13:31:38.543986 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:38.543506    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 22 13:31:38.653696 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:38.653652    2575 generic.go:356] "Generic (PLEG): container finished" podID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerID="aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701" exitCode=0
Jul 22 13:31:38.653905 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:38.653756    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" event={"ID":"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3","Type":"ContainerDied","Data":"aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701"}
Jul 22 13:31:38.654419 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:38.654389    2575 scope.go:117] "RemoveContainer" containerID="aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701"
Jul 22 13:31:38.655664 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:38.655635    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:38.655762 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:38.655724    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:38.655825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:38.655776    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:31:39.544369 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:39.544316    2575 patch_prober.go:28] interesting pod/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 22 13:31:39.544911 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:39.544400    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 22 13:31:39.678096 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:39.678050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" event={"ID":"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3","Type":"ContainerStarted","Data":"b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1"}
Jul 22 13:31:40.069506 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:40.069465    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:40.069723 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:40.069522    2575 patch_prober.go:28] interesting pod/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 22 13:31:40.069723 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:40.069601    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 22 13:31:40.549308 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:40.549267    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:40.574457 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:40.574423    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:40.574892 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:40.574871    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:40.576113 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:40.576095    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:31:41.068265 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:41.068231    2575 patch_prober.go:28] interesting pod/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 22 13:31:41.068455 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:41.068301    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 22 13:31:42.068122 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:42.068084    2575 patch_prober.go:28] interesting pod/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 22 13:31:42.068546 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:42.068139    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 22 13:31:43.068000 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:43.067960    2575 patch_prober.go:28] interesting pod/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 22 13:31:43.068200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:43.068021    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 22 13:31:44.072337 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:44.072302    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:31:44.073357 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:31:44.073340    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:32:09.123876 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:32:09.123845    2575 scope.go:117] "RemoveContainer" containerID="7639779cb6c8ecde0e7a52e2bde63ce103c51f4f90e8ae80546f0e5f2874734e"
Jul 22 13:33:07.787984 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:07.787878    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"]
Jul 22 13:33:07.788584 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:07.788163    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm" podUID="28d92d7f-1bc7-4023-bded-ea31baac12f4" containerName="maas-api" containerID="cri-o://74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2" gracePeriod=30
Jul 22 13:33:08.238048 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.238023    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:33:08.301669 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.301628    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d92d7f-1bc7-4023-bded-ea31baac12f4-maas-api-tls\") pod \"28d92d7f-1bc7-4023-bded-ea31baac12f4\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") "
Jul 22 13:33:08.301838 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.301794    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6vjz\" (UniqueName: \"kubernetes.io/projected/28d92d7f-1bc7-4023-bded-ea31baac12f4-kube-api-access-x6vjz\") pod \"28d92d7f-1bc7-4023-bded-ea31baac12f4\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") "
Jul 22 13:33:08.301838 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.301826    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d92d7f-1bc7-4023-bded-ea31baac12f4-openshift-service-ca\") pod \"28d92d7f-1bc7-4023-bded-ea31baac12f4\" (UID: \"28d92d7f-1bc7-4023-bded-ea31baac12f4\") "
Jul 22 13:33:08.302243 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.302215    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28d92d7f-1bc7-4023-bded-ea31baac12f4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28d92d7f-1bc7-4023-bded-ea31baac12f4" (UID: "28d92d7f-1bc7-4023-bded-ea31baac12f4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:33:08.303784 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.303758    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d92d7f-1bc7-4023-bded-ea31baac12f4-kube-api-access-x6vjz" (OuterVolumeSpecName: "kube-api-access-x6vjz") pod "28d92d7f-1bc7-4023-bded-ea31baac12f4" (UID: "28d92d7f-1bc7-4023-bded-ea31baac12f4"). InnerVolumeSpecName "kube-api-access-x6vjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:33:08.303875 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.303854    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28d92d7f-1bc7-4023-bded-ea31baac12f4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28d92d7f-1bc7-4023-bded-ea31baac12f4" (UID: "28d92d7f-1bc7-4023-bded-ea31baac12f4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:33:08.403543 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.403498    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d92d7f-1bc7-4023-bded-ea31baac12f4-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:08.403543 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.403534    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28d92d7f-1bc7-4023-bded-ea31baac12f4-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:08.403543 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:08.403547    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6vjz\" (UniqueName: \"kubernetes.io/projected/28d92d7f-1bc7-4023-bded-ea31baac12f4-kube-api-access-x6vjz\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:09.105735 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:09.105698    2575 generic.go:356] "Generic (PLEG): container finished" podID="28d92d7f-1bc7-4023-bded-ea31baac12f4" containerID="74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2" exitCode=0
Jul 22 13:33:09.106243 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:09.105762    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"
Jul 22 13:33:09.106243 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:09.105781    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm" event={"ID":"28d92d7f-1bc7-4023-bded-ea31baac12f4","Type":"ContainerDied","Data":"74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2"}
Jul 22 13:33:09.106243 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:09.105848    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm" event={"ID":"28d92d7f-1bc7-4023-bded-ea31baac12f4","Type":"ContainerDied","Data":"119bfcafc79a841ef881a894a5948fb790e55a35a2ab0bbfa613a5d6e3ce993a"}
Jul 22 13:33:09.106243 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:09.105883    2575 scope.go:117] "RemoveContainer" containerID="74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2"
Jul 22 13:33:09.114773 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:09.114752    2575 scope.go:117] "RemoveContainer" containerID="74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2"
Jul 22 13:33:09.115064 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:33:09.115040    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2\": container with ID starting with 74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2 not found: ID does not exist" containerID="74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2"
Jul 22 13:33:09.115137 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:09.115072    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2"} err="failed to get container status \"74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2\": rpc error: code = NotFound desc = could not find container \"74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2\": container with ID starting with 74ca8104307a12a10bccfea3d79813a3f7b8a1773fcde28e008188945c01d9b2 not found: ID does not exist"
Jul 22 13:33:09.125495 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:09.125467    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"]
Jul 22 13:33:09.132737 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:09.132712    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cc03a6-74ccccc588-59wwm"]
Jul 22 13:33:10.947672 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:10.947638    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28d92d7f-1bc7-4023-bded-ea31baac12f4" path="/var/lib/kubelet/pods/28d92d7f-1bc7-4023-bded-ea31baac12f4/volumes"
Jul 22 13:33:19.555682 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:19.555643    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"]
Jul 22 13:33:19.556213 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:19.555865    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy" containerID="cri-o://7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a" gracePeriod=30
Jul 22 13:33:24.805985 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.805959    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:33:24.851429 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851398    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-data\") pod \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") "
Jul 22 13:33:24.851615 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851446    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istiod-ca-cert\") pod \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") "
Jul 22 13:33:24.851615 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851476    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-token\") pod \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") "
Jul 22 13:33:24.851615 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851512    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-podinfo\") pod \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") "
Jul 22 13:33:24.851615 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851553    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8hdb\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-kube-api-access-k8hdb\") pod \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") "
Jul 22 13:33:24.851615 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851569    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-credential-socket\") pod \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") "
Jul 22 13:33:24.851615 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851607    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-envoy\") pod \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") "
Jul 22 13:33:24.851955 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851637    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-socket\") pod \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") "
Jul 22 13:33:24.851955 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851662    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-certs\") pod \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\" (UID: \"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0\") "
Jul 22 13:33:24.851955 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.851793    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" (UID: "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:33:24.852128 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.852045    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istiod-ca-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:24.852246 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.852223    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" (UID: "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:24.852484 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.852358    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" (UID: "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:24.852484 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.852374    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-data" (OuterVolumeSpecName: "istio-data") pod "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" (UID: "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:24.852484 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.852428    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" (UID: "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:24.854079 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.854055    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" (UID: "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:24.854079 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.854073    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-token" (OuterVolumeSpecName: "istio-token") pod "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" (UID: "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:33:24.854226 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.854098    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-kube-api-access-k8hdb" (OuterVolumeSpecName: "kube-api-access-k8hdb") pod "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" (UID: "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0"). InnerVolumeSpecName "kube-api-access-k8hdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:33:24.854226 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.854181    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" (UID: "f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:33:24.952659 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.952622    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-envoy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:24.952659 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.952659    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:24.952862 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.952676    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-workload-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:24.952862 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.952690    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-data\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:24.952862 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.952704    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-token\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:24.952862 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.952716    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-istio-podinfo\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:24.952862 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.952729    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8hdb\" (UniqueName: \"kubernetes.io/projected/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-kube-api-access-k8hdb\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:24.952862 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:24.952744    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0-credential-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:25.181753 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.181648    2575 generic.go:356] "Generic (PLEG): container finished" podID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerID="7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a" exitCode=0
Jul 22 13:33:25.181753 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.181713    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"
Jul 22 13:33:25.182030 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.181743    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" event={"ID":"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0","Type":"ContainerDied","Data":"7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a"}
Jul 22 13:33:25.182030 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.181808    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww" event={"ID":"f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0","Type":"ContainerDied","Data":"16e74c00f87e55c0db885b93424cec93c1fd2a03ca3eb77bd0a1bdc83adda626"}
Jul 22 13:33:25.182030 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.181841    2575 scope.go:117] "RemoveContainer" containerID="7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a"
Jul 22 13:33:25.190633 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.190616    2575 scope.go:117] "RemoveContainer" containerID="9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196"
Jul 22 13:33:25.198192 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.198177    2575 scope.go:117] "RemoveContainer" containerID="7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a"
Jul 22 13:33:25.198523 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:33:25.198492    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a\": container with ID starting with 7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a not found: ID does not exist" containerID="7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a"
Jul 22 13:33:25.198627 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.198535    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a"} err="failed to get container status \"7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a\": rpc error: code = NotFound desc = could not find container \"7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a\": container with ID starting with 7e05b3bf37e9974dd92e0d849982c9482aabef4862dde780e033b7a065eab37a not found: ID does not exist"
Jul 22 13:33:25.198627 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.198560    2575 scope.go:117] "RemoveContainer" containerID="9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196"
Jul 22 13:33:25.198878 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:33:25.198852    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196\": container with ID starting with 9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196 not found: ID does not exist" containerID="9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196"
Jul 22 13:33:25.199005 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.198888    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196"} err="failed to get container status \"9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196\": rpc error: code = NotFound desc = could not find container \"9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196\": container with ID starting with 9a9e1ccc5cd81b4b15a9088b3659d9996604d7e60bc840c5ca2af778320be196 not found: ID does not exist"
Jul 22 13:33:25.201467 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.201448    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"]
Jul 22 13:33:25.205435 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:25.205414    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cc03a6-openshift-default-6d867c4c5c-rsdww"]
Jul 22 13:33:26.951138 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:26.949567    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" path="/var/lib/kubelet/pods/f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0/volumes"
Jul 22 13:33:30.473766 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.473725    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"]
Jul 22 13:33:30.474288 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.474006    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb" podUID="7bad330d-917e-4a99-a40c-0aee46e4d505" containerName="maas-api" containerID="cri-o://b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97" gracePeriod=30
Jul 22 13:33:30.724017 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.723948    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:33:30.807426 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.807391    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc48g\" (UniqueName: \"kubernetes.io/projected/7bad330d-917e-4a99-a40c-0aee46e4d505-kube-api-access-cc48g\") pod \"7bad330d-917e-4a99-a40c-0aee46e4d505\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") "
Jul 22 13:33:30.807603 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.807478    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bad330d-917e-4a99-a40c-0aee46e4d505-maas-api-tls\") pod \"7bad330d-917e-4a99-a40c-0aee46e4d505\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") "
Jul 22 13:33:30.807603 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.807530    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bad330d-917e-4a99-a40c-0aee46e4d505-openshift-service-ca\") pod \"7bad330d-917e-4a99-a40c-0aee46e4d505\" (UID: \"7bad330d-917e-4a99-a40c-0aee46e4d505\") "
Jul 22 13:33:30.807807 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.807786    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bad330d-917e-4a99-a40c-0aee46e4d505-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7bad330d-917e-4a99-a40c-0aee46e4d505" (UID: "7bad330d-917e-4a99-a40c-0aee46e4d505"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:33:30.807910 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.807896    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bad330d-917e-4a99-a40c-0aee46e4d505-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:30.809597 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.809576    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bad330d-917e-4a99-a40c-0aee46e4d505-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7bad330d-917e-4a99-a40c-0aee46e4d505" (UID: "7bad330d-917e-4a99-a40c-0aee46e4d505"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:33:30.809664 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.809641    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bad330d-917e-4a99-a40c-0aee46e4d505-kube-api-access-cc48g" (OuterVolumeSpecName: "kube-api-access-cc48g") pod "7bad330d-917e-4a99-a40c-0aee46e4d505" (UID: "7bad330d-917e-4a99-a40c-0aee46e4d505"). InnerVolumeSpecName "kube-api-access-cc48g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:33:30.909342 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.909305    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc48g\" (UniqueName: \"kubernetes.io/projected/7bad330d-917e-4a99-a40c-0aee46e4d505-kube-api-access-cc48g\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:30.909342 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:30.909337    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bad330d-917e-4a99-a40c-0aee46e4d505-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:31.210454 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:31.210420    2575 generic.go:356] "Generic (PLEG): container finished" podID="7bad330d-917e-4a99-a40c-0aee46e4d505" containerID="b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97" exitCode=0
Jul 22 13:33:31.210621 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:31.210468    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"
Jul 22 13:33:31.210621 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:31.210504    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb" event={"ID":"7bad330d-917e-4a99-a40c-0aee46e4d505","Type":"ContainerDied","Data":"b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97"}
Jul 22 13:33:31.210621 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:31.210564    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb" event={"ID":"7bad330d-917e-4a99-a40c-0aee46e4d505","Type":"ContainerDied","Data":"db2404b3ba76f54d2fd0990922e05eb28e9ec2942ff5784fbdd54eb2aecb5c6f"}
Jul 22 13:33:31.210621 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:31.210586    2575 scope.go:117] "RemoveContainer" containerID="b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97"
Jul 22 13:33:31.221233 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:31.221203    2575 scope.go:117] "RemoveContainer" containerID="b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97"
Jul 22 13:33:31.221486 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:33:31.221466    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97\": container with ID starting with b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97 not found: ID does not exist" containerID="b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97"
Jul 22 13:33:31.221551 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:31.221495    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97"} err="failed to get container status \"b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97\": rpc error: code = NotFound desc = could not find container \"b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97\": container with ID starting with b6f87bcd6ae8680837cfed09adbb49d1e8946fb67078dccbe9dba5b4a23b0b97 not found: ID does not exist"
Jul 22 13:33:31.231570 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:31.231541    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"]
Jul 22 13:33:31.234075 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:31.234054    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-aa3ea2-7684d459c8-bplwb"]
Jul 22 13:33:32.948469 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:32.948430    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bad330d-917e-4a99-a40c-0aee46e4d505" path="/var/lib/kubelet/pods/7bad330d-917e-4a99-a40c-0aee46e4d505/volumes"
Jul 22 13:33:42.295657 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:42.295621    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"]
Jul 22 13:33:42.296166 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:42.295836    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy" containerID="cri-o://b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1" gracePeriod=30
Jul 22 13:33:44.075701 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:44.075664    2575 patch_prober.go:28] interesting pod/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 13:33:44.076094 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:44.075718    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 13:33:46.198424 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198393    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"]
Jul 22 13:33:46.198794 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198773    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bad330d-917e-4a99-a40c-0aee46e4d505" containerName="maas-api"
Jul 22 13:33:46.198794 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198786    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bad330d-917e-4a99-a40c-0aee46e4d505" containerName="maas-api"
Jul 22 13:33:46.198874 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198797    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy"
Jul 22 13:33:46.198874 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198803    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy"
Jul 22 13:33:46.198874 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198811    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28d92d7f-1bc7-4023-bded-ea31baac12f4" containerName="maas-api"
Jul 22 13:33:46.198874 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198816    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28d92d7f-1bc7-4023-bded-ea31baac12f4" containerName="maas-api"
Jul 22 13:33:46.198874 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198827    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy"
Jul 22 13:33:46.198874 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198832    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy"
Jul 22 13:33:46.199112 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198883    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy"
Jul 22 13:33:46.199112 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198891    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="28d92d7f-1bc7-4023-bded-ea31baac12f4" containerName="maas-api"
Jul 22 13:33:46.199112 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198900    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bad330d-917e-4a99-a40c-0aee46e4d505" containerName="maas-api"
Jul 22 13:33:46.199112 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.198912    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f859252f-c9b6-44ef-8bbe-3d19fe3fdbd0" containerName="istio-proxy"
Jul 22 13:33:46.201892 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.201876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.204596 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.204575    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5td92\""
Jul 22 13:33:46.204707 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.204593    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 13:33:46.205754 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.205739    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-609ebe-serving-cert\""
Jul 22 13:33:46.205834 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.205753    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 13:33:46.214026 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.213908    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"]
Jul 22 13:33:46.354145 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.354104    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-openshift-service-ca\") pod \"maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.354353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.354176    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pckd\" (UniqueName: \"kubernetes.io/projected/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-kube-api-access-9pckd\") pod \"maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.354353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.354323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-maas-api-tls\") pod \"maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.455439 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.455351    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-openshift-service-ca\") pod \"maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.455439 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.455396    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pckd\" (UniqueName: \"kubernetes.io/projected/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-kube-api-access-9pckd\") pod \"maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.455634 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.455447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-maas-api-tls\") pod \"maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.456021 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.456000    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-openshift-service-ca\") pod \"maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.457868 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.457845    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-maas-api-tls\") pod \"maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.465196 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.465174    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pckd\" (UniqueName: \"kubernetes.io/projected/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-kube-api-access-9pckd\") pod \"maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.514954 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.514905    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:46.652802 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.652775    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"]
Jul 22 13:33:46.654787 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:33:46.654758    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a2c6db9_261e_4f5c_8e49_e501f55f9ff4.slice/crio-12bdde8fc8f28c65dc1d22e798a7c08dd30dca6b28a27e942211975396fa45bb WatchSource:0}: Error finding container 12bdde8fc8f28c65dc1d22e798a7c08dd30dca6b28a27e942211975396fa45bb: Status 404 returned error can't find the container with id 12bdde8fc8f28c65dc1d22e798a7c08dd30dca6b28a27e942211975396fa45bb
Jul 22 13:33:46.656104 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:46.656085    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 13:33:47.291629 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.291596    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8" event={"ID":"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4","Type":"ContainerStarted","Data":"12bdde8fc8f28c65dc1d22e798a7c08dd30dca6b28a27e942211975396fa45bb"}
Jul 22 13:33:47.563805 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.563783    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:33:47.665772 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.665740    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istiod-ca-cert\") pod \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") "
Jul 22 13:33:47.665984 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.665825    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-podinfo\") pod \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") "
Jul 22 13:33:47.665984 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.665911    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-envoy\") pod \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") "
Jul 22 13:33:47.665984 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.665960    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-socket\") pod \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") "
Jul 22 13:33:47.666166 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.665992    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-data\") pod \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") "
Jul 22 13:33:47.666166 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666057    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-credential-socket\") pod \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") "
Jul 22 13:33:47.666166 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666093    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-token\") pod \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") "
Jul 22 13:33:47.666166 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666122    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr8mk\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-kube-api-access-dr8mk\") pod \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") "
Jul 22 13:33:47.666166 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666156    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-certs\") pod \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\" (UID: \"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3\") "
Jul 22 13:33:47.666415 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666202    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" (UID: "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:33:47.666415 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666255    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" (UID: "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:47.666520 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666468    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:47.666520 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666494    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istiod-ca-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:47.666520 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666463    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" (UID: "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:47.666520 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666507    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" (UID: "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:47.666778 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.666744    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" (UID: "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:47.668078 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.668048    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" (UID: "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:33:47.668078 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.668068    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" (UID: "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:33:47.668515 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.668498    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-kube-api-access-dr8mk" (OuterVolumeSpecName: "kube-api-access-dr8mk") pod "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" (UID: "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3"). InnerVolumeSpecName "kube-api-access-dr8mk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:33:47.668568 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.668547    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" (UID: "9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:33:47.767544 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.767516    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-token\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:47.767544 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.767546    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr8mk\" (UniqueName: \"kubernetes.io/projected/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-kube-api-access-dr8mk\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:47.767735 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.767557    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-workload-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:47.767735 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.767566    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-podinfo\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:47.767735 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.767575    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-envoy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:47.767735 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.767586    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-istio-data\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:47.767735 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:47.767599    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3-credential-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:33:48.297593 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.297561    2575 generic.go:356] "Generic (PLEG): container finished" podID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerID="b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1" exitCode=0
Jul 22 13:33:48.298055 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.297611    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"
Jul 22 13:33:48.298055 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.297652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" event={"ID":"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3","Type":"ContainerDied","Data":"b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1"}
Jul 22 13:33:48.298055 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.297712    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq" event={"ID":"9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3","Type":"ContainerDied","Data":"6f92507878b9f1b801d90fd5ee0f51b64254298689fd88dfbaa99013d1dca861"}
Jul 22 13:33:48.298055 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.297736    2575 scope.go:117] "RemoveContainer" containerID="b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1"
Jul 22 13:33:48.299245 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.299224    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8" event={"ID":"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4","Type":"ContainerStarted","Data":"a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f"}
Jul 22 13:33:48.299359 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.299346    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:48.306948 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.306913    2575 scope.go:117] "RemoveContainer" containerID="aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701"
Jul 22 13:33:48.314661 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.314643    2575 scope.go:117] "RemoveContainer" containerID="b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1"
Jul 22 13:33:48.314904 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:33:48.314885    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1\": container with ID starting with b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1 not found: ID does not exist" containerID="b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1"
Jul 22 13:33:48.314998 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.314912    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1"} err="failed to get container status \"b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1\": rpc error: code = NotFound desc = could not find container \"b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1\": container with ID starting with b9185f396aec30604a82ce2e1d4eabd11d48740caad5d0853350c11f38f284a1 not found: ID does not exist"
Jul 22 13:33:48.314998 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.314941    2575 scope.go:117] "RemoveContainer" containerID="aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701"
Jul 22 13:33:48.315176 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:33:48.315160    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701\": container with ID starting with aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701 not found: ID does not exist" containerID="aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701"
Jul 22 13:33:48.315218 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.315182    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701"} err="failed to get container status \"aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701\": rpc error: code = NotFound desc = could not find container \"aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701\": container with ID starting with aa52a8cc6ba0fb6e0010743cedbef4a02ece3faa11885b958c56507637ab1701 not found: ID does not exist"
Jul 22 13:33:48.324040 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.324001    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8" podStartSLOduration=1.24823365 podStartE2EDuration="2.323989551s" totalImagesPullingTime="1.075755901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:33:48.322504187 +0000 UTC m=+2380.614485231" watchObservedRunningTime="2026-07-22 13:33:48.323989551 +0000 UTC m=+2380.615970584"
Jul 22 13:33:48.337640 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.337614    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"]
Jul 22 13:33:48.343683 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.343662    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-aa3ea2-openshift-default-8df79cd49-ft4nq"]
Jul 22 13:33:48.947757 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:48.947714    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" path="/var/lib/kubelet/pods/9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3/volumes"
Jul 22 13:33:51.442651 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.442610    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"]
Jul 22 13:33:51.443195 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.443180    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy"
Jul 22 13:33:51.443265 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.443198    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy"
Jul 22 13:33:51.443265 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.443208    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy"
Jul 22 13:33:51.443265 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.443217    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy"
Jul 22 13:33:51.443375 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.443315    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy"
Jul 22 13:33:51.443375 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.443333    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dd2d9c9-52c5-44d7-8a90-f5c13ebf24b3" containerName="istio-proxy"
Jul 22 13:33:51.448137 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.448114    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.450668 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.450642    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-3bd5f9-openshift-default-dockercfg-hk2jb\""
Jul 22 13:33:51.450806 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.450667    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 13:33:51.456395 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.456367    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"]
Jul 22 13:33:51.501485 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.501440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-socket\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.501485 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.501479    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-certs\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.501726 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.501537    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77023956-c558-4727-8c88-c36a1c19486b-istio-podinfo\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.501726 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.501570    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-credential-socket\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.501726 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.501589    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-data\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.501726 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.501611    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77023956-c558-4727-8c88-c36a1c19486b-istiod-ca-cert\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.501726 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.501631    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-istio-token\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.501726 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.501663    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-envoy\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.501726 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.501702    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smdcx\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-kube-api-access-smdcx\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.602467 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.602427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-envoy\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.602467 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.602476    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smdcx\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-kube-api-access-smdcx\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.602743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.602612    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-socket\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.602743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.602656    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-certs\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.602743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.602729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77023956-c558-4727-8c88-c36a1c19486b-istio-podinfo\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.602904 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.602791    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-credential-socket\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.602904 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.602824    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-data\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.602904 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.602867    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77023956-c558-4727-8c88-c36a1c19486b-istiod-ca-cert\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.602904 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.602898    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-istio-token\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.603154 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.603021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-socket\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.603154 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.603077    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-certs\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.603154 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.603119    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-credential-socket\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.603301 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.603269    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-data\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.603786 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.603764    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77023956-c558-4727-8c88-c36a1c19486b-istiod-ca-cert\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.604967 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.604923    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-envoy\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.605424 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.605399    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77023956-c558-4727-8c88-c36a1c19486b-istio-podinfo\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.611883 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.611864    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-istio-token\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.612092 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.612072    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smdcx\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-kube-api-access-smdcx\") pod \"e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") " pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.762618 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.762519    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:51.906165 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.906121    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"]
Jul 22 13:33:51.908993 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:33:51.908955    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77023956_c558_4727_8c88_c36a1c19486b.slice/crio-40ed916cdba4b61b87a34d8431daf4e4ef4fdf00d28c9f31260c77e20b2e5ca8 WatchSource:0}: Error finding container 40ed916cdba4b61b87a34d8431daf4e4ef4fdf00d28c9f31260c77e20b2e5ca8: Status 404 returned error can't find the container with id 40ed916cdba4b61b87a34d8431daf4e4ef4fdf00d28c9f31260c77e20b2e5ca8
Jul 22 13:33:51.911208 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.911178    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:33:51.911272 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.911238    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:33:51.911272 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:51.911262    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:33:52.320988 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:52.320955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" event={"ID":"77023956-c558-4727-8c88-c36a1c19486b","Type":"ContainerStarted","Data":"93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999"}
Jul 22 13:33:52.321176 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:52.321008    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" event={"ID":"77023956-c558-4727-8c88-c36a1c19486b","Type":"ContainerStarted","Data":"40ed916cdba4b61b87a34d8431daf4e4ef4fdf00d28c9f31260c77e20b2e5ca8"}
Jul 22 13:33:52.342739 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:52.342689    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" podStartSLOduration=1.34267089 podStartE2EDuration="1.34267089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:33:52.341263047 +0000 UTC m=+2384.633244080" watchObservedRunningTime="2026-07-22 13:33:52.34267089 +0000 UTC m=+2384.634651922"
Jul 22 13:33:52.763755 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:52.763658    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:52.768458 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:52.768432    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:53.326182 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:53.326150    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:53.327337 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:53.327312    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:33:54.310026 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.309998    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:33:54.557305 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.557270    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"]
Jul 22 13:33:54.561545 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.561486    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:54.564572 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.564550    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-3bd5f9-serving-cert\""
Jul 22 13:33:54.573476 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.573447    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"]
Jul 22 13:33:54.630561 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.630526    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-maas-api-tls\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:54.630725 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.630583    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-openshift-service-ca\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:54.630725 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.630604    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjbgv\" (UniqueName: \"kubernetes.io/projected/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-kube-api-access-jjbgv\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:54.731175 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.731139    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-maas-api-tls\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:54.731351 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.731199    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-openshift-service-ca\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:54.731351 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.731224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjbgv\" (UniqueName: \"kubernetes.io/projected/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-kube-api-access-jjbgv\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:54.731351 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:33:54.731333    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-serving-cert: secret "maas-api-e2e-api-b-3bd5f9-serving-cert" not found
Jul 22 13:33:54.731549 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:33:54.731399    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-maas-api-tls podName:dd64b68f-fd47-4bd8-938d-6d19ee65b1e7 nodeName:}" failed. No retries permitted until 2026-07-22 13:33:55.231375914 +0000 UTC m=+2387.523356930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-maas-api-tls") pod "maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz" (UID: "dd64b68f-fd47-4bd8-938d-6d19ee65b1e7") : secret "maas-api-e2e-api-b-3bd5f9-serving-cert" not found
Jul 22 13:33:54.732079 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.732057    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-openshift-service-ca\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:54.742267 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:54.742241    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjbgv\" (UniqueName: \"kubernetes.io/projected/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-kube-api-access-jjbgv\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:55.236981 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:55.236947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-maas-api-tls\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:55.239309 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:55.239285    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-maas-api-tls\") pod \"maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:55.474683 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:55.474651    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:55.614083 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:55.614059    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"]
Jul 22 13:33:55.615886 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:33:55.615860    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd64b68f_fd47_4bd8_938d_6d19ee65b1e7.slice/crio-6c762eb4112ba5f78f1a9385e946238df7658487c3dd6fdd33e3abacf2a63a90 WatchSource:0}: Error finding container 6c762eb4112ba5f78f1a9385e946238df7658487c3dd6fdd33e3abacf2a63a90: Status 404 returned error can't find the container with id 6c762eb4112ba5f78f1a9385e946238df7658487c3dd6fdd33e3abacf2a63a90
Jul 22 13:33:56.344357 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:56.344316    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz" event={"ID":"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7","Type":"ContainerStarted","Data":"c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856"}
Jul 22 13:33:56.344523 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:56.344422    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz" event={"ID":"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7","Type":"ContainerStarted","Data":"6c762eb4112ba5f78f1a9385e946238df7658487c3dd6fdd33e3abacf2a63a90"}
Jul 22 13:33:56.344523 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:56.344452    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:33:56.364629 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:33:56.364576    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz" podStartSLOduration=1.8633647949999999 podStartE2EDuration="2.364561231s" totalImagesPullingTime="501.196436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:33:56.363496227 +0000 UTC m=+2388.655477272" watchObservedRunningTime="2026-07-22 13:33:56.364561231 +0000 UTC m=+2388.656542263"
Jul 22 13:34:02.356362 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:02.356335    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:34:08.328009 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:08.327969    2575 patch_prober.go:28] interesting pod/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:34:08.328500 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:08.328028    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 22 13:34:08.406025 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:08.405914    2575 generic.go:356] "Generic (PLEG): container finished" podID="77023956-c558-4727-8c88-c36a1c19486b" containerID="93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999" exitCode=0
Jul 22 13:34:08.406025 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:08.405992    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" event={"ID":"77023956-c558-4727-8c88-c36a1c19486b","Type":"ContainerDied","Data":"93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999"}
Jul 22 13:34:08.406427 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:08.406411    2575 scope.go:117] "RemoveContainer" containerID="93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999"
Jul 22 13:34:08.407556 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:08.407529    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:34:08.407647 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:08.407610    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:34:08.407710 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:08.407656    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:34:08.997103 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:08.997070    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:34:09.003793 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:09.003769    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:34:09.155005 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:09.154973    2575 scope.go:117] "RemoveContainer" containerID="64d996be5ab16ce67cc8c1614aca3c5eebcef09d8a4dbf048ecc6dc2a52314f6"
Jul 22 13:34:09.412353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:09.412322    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" event={"ID":"77023956-c558-4727-8c88-c36a1c19486b","Type":"ContainerStarted","Data":"2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126"}
Jul 22 13:34:09.763270 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:09.763181    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:34:09.763451 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:09.763433    2575 patch_prober.go:28] interesting pod/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:34:09.763495 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:09.763475    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 22 13:34:10.763302 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:10.763262    2575 patch_prober.go:28] interesting pod/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:34:10.763702 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:10.763320    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 22 13:34:11.763054 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:11.763011    2575 patch_prober.go:28] interesting pod/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:34:11.763241 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:11.763076    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 22 13:34:12.763691 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:12.763648    2575 patch_prober.go:28] interesting pod/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:34:12.764146 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:12.763712    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 22 13:34:13.767042 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:13.767010    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:34:13.767429 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:13.767380    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:34:13.768140 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:13.768119    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:34:23.056204 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.056168    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"]
Jul 22 13:34:23.056786 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.056418    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8" podUID="0a2c6db9-261e-4f5c-8e49-e501f55f9ff4" containerName="maas-api" containerID="cri-o://a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f" gracePeriod=30
Jul 22 13:34:23.304665 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.304638    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:34:23.398886 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.398848    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-openshift-service-ca\") pod \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") "
Jul 22 13:34:23.399097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.398967    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-maas-api-tls\") pod \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") "
Jul 22 13:34:23.399097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.399037    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pckd\" (UniqueName: \"kubernetes.io/projected/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-kube-api-access-9pckd\") pod \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\" (UID: \"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4\") "
Jul 22 13:34:23.399272 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.399248    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0a2c6db9-261e-4f5c-8e49-e501f55f9ff4" (UID: "0a2c6db9-261e-4f5c-8e49-e501f55f9ff4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:34:23.401105 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.401070    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-kube-api-access-9pckd" (OuterVolumeSpecName: "kube-api-access-9pckd") pod "0a2c6db9-261e-4f5c-8e49-e501f55f9ff4" (UID: "0a2c6db9-261e-4f5c-8e49-e501f55f9ff4"). InnerVolumeSpecName "kube-api-access-9pckd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:34:23.401219 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.401160    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0a2c6db9-261e-4f5c-8e49-e501f55f9ff4" (UID: "0a2c6db9-261e-4f5c-8e49-e501f55f9ff4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:34:23.483425 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.483390    2575 generic.go:356] "Generic (PLEG): container finished" podID="0a2c6db9-261e-4f5c-8e49-e501f55f9ff4" containerID="a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f" exitCode=0
Jul 22 13:34:23.483597 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.483439    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"
Jul 22 13:34:23.483597 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.483459    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8" event={"ID":"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4","Type":"ContainerDied","Data":"a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f"}
Jul 22 13:34:23.483597 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.483514    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8" event={"ID":"0a2c6db9-261e-4f5c-8e49-e501f55f9ff4","Type":"ContainerDied","Data":"12bdde8fc8f28c65dc1d22e798a7c08dd30dca6b28a27e942211975396fa45bb"}
Jul 22 13:34:23.483597 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.483544    2575 scope.go:117] "RemoveContainer" containerID="a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f"
Jul 22 13:34:23.492538 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.492517    2575 scope.go:117] "RemoveContainer" containerID="a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f"
Jul 22 13:34:23.492797 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:34:23.492779    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f\": container with ID starting with a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f not found: ID does not exist" containerID="a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f"
Jul 22 13:34:23.492858 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.492805    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f"} err="failed to get container status \"a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f\": rpc error: code = NotFound desc = could not find container \"a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f\": container with ID starting with a917d348032a673bd0fd53ea93fc09b81d40fc77e09e21d3e8ff06fd33ab507f not found: ID does not exist"
Jul 22 13:34:23.500053 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.500033    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pckd\" (UniqueName: \"kubernetes.io/projected/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-kube-api-access-9pckd\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:34:23.500115 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.500055    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:34:23.500115 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.500064    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:34:23.507162 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.507137    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"]
Jul 22 13:34:23.511835 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:23.511811    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-609ebe-5666ddd6fb-52jt8"]
Jul 22 13:34:24.947677 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:24.947642    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a2c6db9-261e-4f5c-8e49-e501f55f9ff4" path="/var/lib/kubelet/pods/0a2c6db9-261e-4f5c-8e49-e501f55f9ff4/volumes"
Jul 22 13:34:34.857531 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:34.857444    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"]
Jul 22 13:34:34.857961 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:34.857853    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2c6db9-261e-4f5c-8e49-e501f55f9ff4" containerName="maas-api"
Jul 22 13:34:34.857961 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:34.857867    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2c6db9-261e-4f5c-8e49-e501f55f9ff4" containerName="maas-api"
Jul 22 13:34:34.858055 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:34.857988    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a2c6db9-261e-4f5c-8e49-e501f55f9ff4" containerName="maas-api"
Jul 22 13:34:34.862771 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:34.862750    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"
Jul 22 13:34:34.865509 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:34.865489    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:34:34.869609 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:34.869583    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"]
Jul 22 13:34:34.899520 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:34.899491    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c4e17-1ef3-4fea-89fc-f269c5c5f083-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns\" (UID: \"771c4e17-1ef3-4fea-89fc-f269c5c5f083\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"
Jul 22 13:34:35.000344 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:35.000304    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c4e17-1ef3-4fea-89fc-f269c5c5f083-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns\" (UID: \"771c4e17-1ef3-4fea-89fc-f269c5c5f083\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"
Jul 22 13:34:35.000986 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:35.000965    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c4e17-1ef3-4fea-89fc-f269c5c5f083-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns\" (UID: \"771c4e17-1ef3-4fea-89fc-f269c5c5f083\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"
Jul 22 13:34:35.175101 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:35.175015    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"
Jul 22 13:34:35.307767 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:35.307742    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"]
Jul 22 13:34:35.309735 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:34:35.309708    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771c4e17_1ef3_4fea_89fc_f269c5c5f083.slice/crio-e2236938f54f9df88994e91f24180c032e551a4dbe73abc708c3b451bf5c972e WatchSource:0}: Error finding container e2236938f54f9df88994e91f24180c032e551a4dbe73abc708c3b451bf5c972e: Status 404 returned error can't find the container with id e2236938f54f9df88994e91f24180c032e551a4dbe73abc708c3b451bf5c972e
Jul 22 13:34:35.546985 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:35.546955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns" event={"ID":"771c4e17-1ef3-4fea-89fc-f269c5c5f083","Type":"ContainerStarted","Data":"e2236938f54f9df88994e91f24180c032e551a4dbe73abc708c3b451bf5c972e"}
Jul 22 13:34:36.553601 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:36.553514    2575 generic.go:356] "Generic (PLEG): container finished" podID="771c4e17-1ef3-4fea-89fc-f269c5c5f083" containerID="cc6babec15f25eb8e110116e6f432b4ede6a65cf91e7c27024d14315371e30c0" exitCode=0
Jul 22 13:34:36.553969 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:36.553597    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns" event={"ID":"771c4e17-1ef3-4fea-89fc-f269c5c5f083","Type":"ContainerDied","Data":"cc6babec15f25eb8e110116e6f432b4ede6a65cf91e7c27024d14315371e30c0"}
Jul 22 13:34:37.696555 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:37.696528    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"
Jul 22 13:34:37.725881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:37.725850    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c4e17-1ef3-4fea-89fc-f269c5c5f083-maas-api-service-ca\") pod \"771c4e17-1ef3-4fea-89fc-f269c5c5f083\" (UID: \"771c4e17-1ef3-4fea-89fc-f269c5c5f083\") "
Jul 22 13:34:37.726208 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:37.726187    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/771c4e17-1ef3-4fea-89fc-f269c5c5f083-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "771c4e17-1ef3-4fea-89fc-f269c5c5f083" (UID: "771c4e17-1ef3-4fea-89fc-f269c5c5f083"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:34:37.827200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:37.827162    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c4e17-1ef3-4fea-89fc-f269c5c5f083-maas-api-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:34:38.564882 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:38.564853    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"
Jul 22 13:34:38.565097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:38.564891    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns" event={"ID":"771c4e17-1ef3-4fea-89fc-f269c5c5f083","Type":"ContainerDied","Data":"e2236938f54f9df88994e91f24180c032e551a4dbe73abc708c3b451bf5c972e"}
Jul 22 13:34:38.565097 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:38.564968    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2236938f54f9df88994e91f24180c032e551a4dbe73abc708c3b451bf5c972e"
Jul 22 13:34:50.306320 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.306285    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"]
Jul 22 13:34:50.306827 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.306508    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz" podUID="dd64b68f-fd47-4bd8-938d-6d19ee65b1e7" containerName="maas-api" containerID="cri-o://c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856" gracePeriod=30
Jul 22 13:34:50.561562 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.561491    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:34:50.623290 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.623256    2575 generic.go:356] "Generic (PLEG): container finished" podID="dd64b68f-fd47-4bd8-938d-6d19ee65b1e7" containerID="c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856" exitCode=0
Jul 22 13:34:50.623494 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.623298    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"
Jul 22 13:34:50.623494 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.623319    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz" event={"ID":"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7","Type":"ContainerDied","Data":"c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856"}
Jul 22 13:34:50.623494 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.623371    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz" event={"ID":"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7","Type":"ContainerDied","Data":"6c762eb4112ba5f78f1a9385e946238df7658487c3dd6fdd33e3abacf2a63a90"}
Jul 22 13:34:50.623494 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.623399    2575 scope.go:117] "RemoveContainer" containerID="c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856"
Jul 22 13:34:50.632530 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.632508    2575 scope.go:117] "RemoveContainer" containerID="c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856"
Jul 22 13:34:50.632806 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:34:50.632786    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856\": container with ID starting with c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856 not found: ID does not exist" containerID="c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856"
Jul 22 13:34:50.632879 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.632820    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856"} err="failed to get container status \"c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856\": rpc error: code = NotFound desc = could not find container \"c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856\": container with ID starting with c7f421a4e6970c6a9a34769786afb353b629bd9c6c6056c254231727c55b1856 not found: ID does not exist"
Jul 22 13:34:50.641115 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.641093    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjbgv\" (UniqueName: \"kubernetes.io/projected/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-kube-api-access-jjbgv\") pod \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") "
Jul 22 13:34:50.641223 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.641132    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-maas-api-tls\") pod \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") "
Jul 22 13:34:50.641272 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.641234    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-openshift-service-ca\") pod \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\" (UID: \"dd64b68f-fd47-4bd8-938d-6d19ee65b1e7\") "
Jul 22 13:34:50.641580 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.641558    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dd64b68f-fd47-4bd8-938d-6d19ee65b1e7" (UID: "dd64b68f-fd47-4bd8-938d-6d19ee65b1e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:34:50.643196 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.643172    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-kube-api-access-jjbgv" (OuterVolumeSpecName: "kube-api-access-jjbgv") pod "dd64b68f-fd47-4bd8-938d-6d19ee65b1e7" (UID: "dd64b68f-fd47-4bd8-938d-6d19ee65b1e7"). InnerVolumeSpecName "kube-api-access-jjbgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:34:50.643281 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.643193    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dd64b68f-fd47-4bd8-938d-6d19ee65b1e7" (UID: "dd64b68f-fd47-4bd8-938d-6d19ee65b1e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:34:50.742711 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.742660    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:34:50.742711 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.742704    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjbgv\" (UniqueName: \"kubernetes.io/projected/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-kube-api-access-jjbgv\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:34:50.742711 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.742715    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:34:50.951041 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.951010    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"]
Jul 22 13:34:50.961107 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:50.961081    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3bd5f9-7f6cfb8dbf-fl2zz"]
Jul 22 13:34:52.948181 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:52.948148    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd64b68f-fd47-4bd8-938d-6d19ee65b1e7" path="/var/lib/kubelet/pods/dd64b68f-fd47-4bd8-938d-6d19ee65b1e7/volumes"
Jul 22 13:34:54.948838 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:54.948805    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"]
Jul 22 13:34:54.948838 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:54.948842    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3bd5f9-ffvns"]
Jul 22 13:34:56.948333 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:34:56.948298    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="771c4e17-1ef3-4fea-89fc-f269c5c5f083" path="/var/lib/kubelet/pods/771c4e17-1ef3-4fea-89fc-f269c5c5f083/volumes"
Jul 22 13:35:02.032290 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.032260    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"]
Jul 22 13:35:02.032731 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.032527    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy" containerID="cri-o://2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126" gracePeriod=30
Jul 22 13:35:02.949641 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.949610    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"]
Jul 22 13:35:02.949984 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.949969    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="771c4e17-1ef3-4fea-89fc-f269c5c5f083" containerName="revoke-keys"
Jul 22 13:35:02.949984 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.949984    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="771c4e17-1ef3-4fea-89fc-f269c5c5f083" containerName="revoke-keys"
Jul 22 13:35:02.950155 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.950001    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd64b68f-fd47-4bd8-938d-6d19ee65b1e7" containerName="maas-api"
Jul 22 13:35:02.950155 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.950007    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd64b68f-fd47-4bd8-938d-6d19ee65b1e7" containerName="maas-api"
Jul 22 13:35:02.950155 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.950072    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd64b68f-fd47-4bd8-938d-6d19ee65b1e7" containerName="maas-api"
Jul 22 13:35:02.950155 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.950081    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="771c4e17-1ef3-4fea-89fc-f269c5c5f083" containerName="revoke-keys"
Jul 22 13:35:02.954433 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.954415    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:02.957080 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.957059    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-15ce38-openshift-default-dockercfg-cjsbb\""
Jul 22 13:35:02.965565 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:02.965538    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"]
Jul 22 13:35:03.053955 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.053902    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-data\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.053955 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.053957    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fb5c\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-kube-api-access-9fb5c\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.054514 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.054015    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-certs\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.054514 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.054067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-token\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.054514 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.054095    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18f7028c-4287-4e1b-a47b-816fa2d9462e-istiod-ca-cert\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.054514 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.054172    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-socket\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.054514 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.054187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-credential-socket\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.054514 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.054208    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-podinfo\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.054514 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.054225    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-envoy\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155334 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-data\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155334 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155333    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fb5c\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-kube-api-access-9fb5c\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-certs\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155385    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-token\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155409    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18f7028c-4287-4e1b-a47b-816fa2d9462e-istiod-ca-cert\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155488    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-socket\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155511    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-credential-socket\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-podinfo\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155577 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-envoy\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155962 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155775    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-certs\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155962 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155873    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-credential-socket\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.155962 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.155888    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-socket\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.156142 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.156008    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-data\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.156327 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.156304    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18f7028c-4287-4e1b-a47b-816fa2d9462e-istiod-ca-cert\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.157952 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.157897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-envoy\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.158143 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.158124    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-podinfo\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.164471 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.164444    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-token\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.165079 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.165058    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fb5c\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-kube-api-access-9fb5c\") pod \"e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") " pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.266164 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.266073    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:03.403859 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.403830    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"]
Jul 22 13:35:03.405868 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:35:03.405840    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f7028c_4287_4e1b_a47b_816fa2d9462e.slice/crio-f95248fa6f19e24f7b1de127717e00f46b891a4137eb97412a68c9ce518ff4c4 WatchSource:0}: Error finding container f95248fa6f19e24f7b1de127717e00f46b891a4137eb97412a68c9ce518ff4c4: Status 404 returned error can't find the container with id f95248fa6f19e24f7b1de127717e00f46b891a4137eb97412a68c9ce518ff4c4
Jul 22 13:35:03.408609 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.408573    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:03.408702 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.408662    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:03.408743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.408705    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:03.688644 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.688604    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" event={"ID":"18f7028c-4287-4e1b-a47b-816fa2d9462e","Type":"ContainerStarted","Data":"a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d"}
Jul 22 13:35:03.688854 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.688663    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" event={"ID":"18f7028c-4287-4e1b-a47b-816fa2d9462e","Type":"ContainerStarted","Data":"f95248fa6f19e24f7b1de127717e00f46b891a4137eb97412a68c9ce518ff4c4"}
Jul 22 13:35:03.717295 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:03.717247    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" podStartSLOduration=1.7172308059999999 podStartE2EDuration="1.717230806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:35:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:35:03.716563492 +0000 UTC m=+2456.008544566" watchObservedRunningTime="2026-07-22 13:35:03.717230806 +0000 UTC m=+2456.009211839"
Jul 22 13:35:04.266879 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:04.266839    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:04.271839 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:04.271816    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:04.696566 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:04.696535    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:04.697617 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:04.697600    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:06.002421 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.002381    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"]
Jul 22 13:35:06.006377 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.006355    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"]
Jul 22 13:35:06.006516 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.006443    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.008878 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.008850    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-15ce38-serving-cert\""
Jul 22 13:35:06.009155 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.009130    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 13:35:06.009267 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.009218    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5td92\""
Jul 22 13:35:06.009323 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.009309    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 13:35:06.085530 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.085500    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx5dq\" (UniqueName: \"kubernetes.io/projected/24ca63b4-63cc-48a3-946d-d1742244bf03-kube-api-access-cx5dq\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.085530 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.085543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24ca63b4-63cc-48a3-946d-d1742244bf03-maas-api-tls\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.085774 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.085642    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24ca63b4-63cc-48a3-946d-d1742244bf03-openshift-service-ca\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.186867 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.186830    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24ca63b4-63cc-48a3-946d-d1742244bf03-openshift-service-ca\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.187087 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.186914    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx5dq\" (UniqueName: \"kubernetes.io/projected/24ca63b4-63cc-48a3-946d-d1742244bf03-kube-api-access-cx5dq\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.187087 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.186950    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24ca63b4-63cc-48a3-946d-d1742244bf03-maas-api-tls\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.187087 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:35:06.187062    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-serving-cert: secret "maas-api-e2e-shared-a-15ce38-serving-cert" not found
Jul 22 13:35:06.187268 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:35:06.187115    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ca63b4-63cc-48a3-946d-d1742244bf03-maas-api-tls podName:24ca63b4-63cc-48a3-946d-d1742244bf03 nodeName:}" failed. No retries permitted until 2026-07-22 13:35:06.687099368 +0000 UTC m=+2458.979080378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/24ca63b4-63cc-48a3-946d-d1742244bf03-maas-api-tls") pod "maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75" (UID: "24ca63b4-63cc-48a3-946d-d1742244bf03") : secret "maas-api-e2e-shared-a-15ce38-serving-cert" not found
Jul 22 13:35:06.187578 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.187554    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24ca63b4-63cc-48a3-946d-d1742244bf03-openshift-service-ca\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.199426 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.199398    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx5dq\" (UniqueName: \"kubernetes.io/projected/24ca63b4-63cc-48a3-946d-d1742244bf03-kube-api-access-cx5dq\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.692034 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.691999    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24ca63b4-63cc-48a3-946d-d1742244bf03-maas-api-tls\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.694716 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.694684    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24ca63b4-63cc-48a3-946d-d1742244bf03-maas-api-tls\") pod \"maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:06.919081 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:06.919034    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:07.067991 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.067909    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"]
Jul 22 13:35:07.111963 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:35:07.111913    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ca63b4_63cc_48a3_946d_d1742244bf03.slice/crio-9b65c797760f7153c4cf9f94167069809ce63b77f7b5b7b1e7bff4f4b23f5124 WatchSource:0}: Error finding container 9b65c797760f7153c4cf9f94167069809ce63b77f7b5b7b1e7bff4f4b23f5124: Status 404 returned error can't find the container with id 9b65c797760f7153c4cf9f94167069809ce63b77f7b5b7b1e7bff4f4b23f5124
Jul 22 13:35:07.280644 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.280619    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:35:07.399372 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399337    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-envoy\") pod \"77023956-c558-4727-8c88-c36a1c19486b\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") "
Jul 22 13:35:07.399372 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399381    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smdcx\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-kube-api-access-smdcx\") pod \"77023956-c558-4727-8c88-c36a1c19486b\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") "
Jul 22 13:35:07.399650 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399405    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77023956-c558-4727-8c88-c36a1c19486b-istio-podinfo\") pod \"77023956-c558-4727-8c88-c36a1c19486b\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") "
Jul 22 13:35:07.399650 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399422    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-credential-socket\") pod \"77023956-c558-4727-8c88-c36a1c19486b\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") "
Jul 22 13:35:07.399650 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399625    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77023956-c558-4727-8c88-c36a1c19486b-istiod-ca-cert\") pod \"77023956-c558-4727-8c88-c36a1c19486b\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") "
Jul 22 13:35:07.399825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399668    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-socket\") pod \"77023956-c558-4727-8c88-c36a1c19486b\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") "
Jul 22 13:35:07.399825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399713    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-certs\") pod \"77023956-c558-4727-8c88-c36a1c19486b\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") "
Jul 22 13:35:07.399825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399743    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77023956-c558-4727-8c88-c36a1c19486b" (UID: "77023956-c558-4727-8c88-c36a1c19486b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:35:07.399825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399762    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-data\") pod \"77023956-c558-4727-8c88-c36a1c19486b\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") "
Jul 22 13:35:07.399825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.399786    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-istio-token\") pod \"77023956-c558-4727-8c88-c36a1c19486b\" (UID: \"77023956-c558-4727-8c88-c36a1c19486b\") "
Jul 22 13:35:07.400132 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.400006    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77023956-c558-4727-8c88-c36a1c19486b" (UID: "77023956-c558-4727-8c88-c36a1c19486b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:35:07.400132 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.400029    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77023956-c558-4727-8c88-c36a1c19486b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77023956-c558-4727-8c88-c36a1c19486b" (UID: "77023956-c558-4727-8c88-c36a1c19486b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:35:07.400249 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.400172    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77023956-c558-4727-8c88-c36a1c19486b-istiod-ca-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:35:07.400249 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.400193    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:35:07.400249 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.400207    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-credential-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:35:07.400426 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.400286    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77023956-c558-4727-8c88-c36a1c19486b" (UID: "77023956-c558-4727-8c88-c36a1c19486b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:35:07.400638 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.400617    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-data" (OuterVolumeSpecName: "istio-data") pod "77023956-c558-4727-8c88-c36a1c19486b" (UID: "77023956-c558-4727-8c88-c36a1c19486b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:35:07.401779 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.401748    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-kube-api-access-smdcx" (OuterVolumeSpecName: "kube-api-access-smdcx") pod "77023956-c558-4727-8c88-c36a1c19486b" (UID: "77023956-c558-4727-8c88-c36a1c19486b"). InnerVolumeSpecName "kube-api-access-smdcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:35:07.401897 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.401749    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77023956-c558-4727-8c88-c36a1c19486b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77023956-c558-4727-8c88-c36a1c19486b" (UID: "77023956-c558-4727-8c88-c36a1c19486b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:35:07.402014 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.401993    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77023956-c558-4727-8c88-c36a1c19486b" (UID: "77023956-c558-4727-8c88-c36a1c19486b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:35:07.402192 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.402173    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-istio-token" (OuterVolumeSpecName: "istio-token") pod "77023956-c558-4727-8c88-c36a1c19486b" (UID: "77023956-c558-4727-8c88-c36a1c19486b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:35:07.501412 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.501366    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-envoy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:35:07.501412 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.501404    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smdcx\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-kube-api-access-smdcx\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:35:07.501412 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.501414    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77023956-c558-4727-8c88-c36a1c19486b-istio-podinfo\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:35:07.501412 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.501422    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-workload-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:35:07.501677 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.501430    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77023956-c558-4727-8c88-c36a1c19486b-istio-data\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:35:07.501677 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.501438    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77023956-c558-4727-8c88-c36a1c19486b-istio-token\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:35:07.711987 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.711952    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75" event={"ID":"24ca63b4-63cc-48a3-946d-d1742244bf03","Type":"ContainerStarted","Data":"9b65c797760f7153c4cf9f94167069809ce63b77f7b5b7b1e7bff4f4b23f5124"}
Jul 22 13:35:07.713574 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.713542    2575 generic.go:356] "Generic (PLEG): container finished" podID="77023956-c558-4727-8c88-c36a1c19486b" containerID="2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126" exitCode=0
Jul 22 13:35:07.713710 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.713596    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"
Jul 22 13:35:07.713710 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.713626    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" event={"ID":"77023956-c558-4727-8c88-c36a1c19486b","Type":"ContainerDied","Data":"2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126"}
Jul 22 13:35:07.713710 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.713687    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r" event={"ID":"77023956-c558-4727-8c88-c36a1c19486b","Type":"ContainerDied","Data":"40ed916cdba4b61b87a34d8431daf4e4ef4fdf00d28c9f31260c77e20b2e5ca8"}
Jul 22 13:35:07.713879 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.713717    2575 scope.go:117] "RemoveContainer" containerID="2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126"
Jul 22 13:35:07.723686 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.723654    2575 scope.go:117] "RemoveContainer" containerID="93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999"
Jul 22 13:35:07.738160 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.738133    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"]
Jul 22 13:35:07.741382 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.741359    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-3bd5f9-openshift-default-74d6dbb467-nr77r"]
Jul 22 13:35:07.745108 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.745088    2575 scope.go:117] "RemoveContainer" containerID="2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126"
Jul 22 13:35:07.745437 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:35:07.745414    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126\": container with ID starting with 2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126 not found: ID does not exist" containerID="2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126"
Jul 22 13:35:07.745496 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.745447    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126"} err="failed to get container status \"2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126\": rpc error: code = NotFound desc = could not find container \"2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126\": container with ID starting with 2dee5821e0b8097edda8f400e01df703788685aee33c51b9fb6e369825420126 not found: ID does not exist"
Jul 22 13:35:07.745496 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.745468    2575 scope.go:117] "RemoveContainer" containerID="93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999"
Jul 22 13:35:07.745744 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:35:07.745726    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999\": container with ID starting with 93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999 not found: ID does not exist" containerID="93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999"
Jul 22 13:35:07.745844 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:07.745749    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999"} err="failed to get container status \"93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999\": rpc error: code = NotFound desc = could not find container \"93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999\": container with ID starting with 93b02bfac9a047f4e9036810cc94a926c9050412163fe2f6122bce8d51e04999 not found: ID does not exist"
Jul 22 13:35:08.721993 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:08.721953    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75" event={"ID":"24ca63b4-63cc-48a3-946d-d1742244bf03","Type":"ContainerStarted","Data":"93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470"}
Jul 22 13:35:08.722434 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:08.722089    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:08.742144 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:08.742097    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75" podStartSLOduration=3.194556608 podStartE2EDuration="3.742081298s" totalImagesPullingTime="547.52469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:35:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:35:08.740339249 +0000 UTC m=+2461.032320305" watchObservedRunningTime="2026-07-22 13:35:08.742081298 +0000 UTC m=+2461.034062330"
Jul 22 13:35:08.949241 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:08.949207    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77023956-c558-4727-8c88-c36a1c19486b" path="/var/lib/kubelet/pods/77023956-c558-4727-8c88-c36a1c19486b/volumes"
Jul 22 13:35:09.185450 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:09.185422    2575 scope.go:117] "RemoveContainer" containerID="583d7b91510025ee1336b1cbfbef26e506fe7b54b47e29287a8dde6cef18ee70"
Jul 22 13:35:10.883158 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:10.883125    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"]
Jul 22 13:35:10.883547 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:10.883533    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy"
Jul 22 13:35:10.883547 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:10.883548    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy"
Jul 22 13:35:10.883653 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:10.883618    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy"
Jul 22 13:35:10.888223 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:10.888201    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:10.891505 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:10.891485    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-19e216-openshift-default-dockercfg-g454h\""
Jul 22 13:35:10.901013 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:10.900990    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"]
Jul 22 13:35:11.032844 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.032808    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-credential-socket\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.032844 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.032842    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-envoy\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.033127 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.032864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-token\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.033127 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.032886    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gc45\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-kube-api-access-4gc45\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.033127 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.032988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istiod-ca-cert\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.033127 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.033033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-data\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.033127 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.033105    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-podinfo\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.033310 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.033158    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-socket\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.033310 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.033195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-certs\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.134529 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134435    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-podinfo\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.134529 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134489    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-socket\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.134529 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-certs\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.134825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134543    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-credential-socket\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.134825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-envoy\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.134825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134581    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-token\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.134825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gc45\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-kube-api-access-4gc45\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.134825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134637    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istiod-ca-cert\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.134825 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134673    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-data\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.135170 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.134988    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-certs\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.135170 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.135030    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-credential-socket\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.135170 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.135060    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-data\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.135170 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.135107    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-socket\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.135542 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.135519    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istiod-ca-cert\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.137057 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.137036    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-envoy\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.137193 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.137177    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-podinfo\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.147481 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.147460    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gc45\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-kube-api-access-4gc45\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.147586 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.147550    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-token\") pod \"e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") " pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.199766 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.199731    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:11.346668 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.346640    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"]
Jul 22 13:35:11.348568 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:35:11.348538    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d4cb7b0_f8e2_4d87_b2e6_10b475e904d5.slice/crio-7539892b2786642829a114352d38cb69cdf4d9d13b8db7eecbc0c8f71e98a501 WatchSource:0}: Error finding container 7539892b2786642829a114352d38cb69cdf4d9d13b8db7eecbc0c8f71e98a501: Status 404 returned error can't find the container with id 7539892b2786642829a114352d38cb69cdf4d9d13b8db7eecbc0c8f71e98a501
Jul 22 13:35:11.350739 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.350703    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:11.350862 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.350791    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:11.350862 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.350838    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:11.745312 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.745211    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" event={"ID":"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5","Type":"ContainerStarted","Data":"b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0"}
Jul 22 13:35:11.746441 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.745558    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" event={"ID":"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5","Type":"ContainerStarted","Data":"7539892b2786642829a114352d38cb69cdf4d9d13b8db7eecbc0c8f71e98a501"}
Jul 22 13:35:11.767582 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:11.767527    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" podStartSLOduration=1.7675115099999998 podStartE2EDuration="1.76751151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:35:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:35:11.766014673 +0000 UTC m=+2464.057995729" watchObservedRunningTime="2026-07-22 13:35:11.76751151 +0000 UTC m=+2464.059492543"
Jul 22 13:35:12.200649 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:12.200610    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:12.205580 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:12.205555    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:12.752977 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:12.752947    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:12.754112 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:12.754094    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:13.866909 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.866879    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"]
Jul 22 13:35:13.867379 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.867290    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy"
Jul 22 13:35:13.867379 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.867303    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy"
Jul 22 13:35:13.867379 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.867359    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="77023956-c558-4727-8c88-c36a1c19486b" containerName="istio-proxy"
Jul 22 13:35:13.877021 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.876434    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:13.880718 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.880691    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-19e216-serving-cert\""
Jul 22 13:35:13.886210 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.886161    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"]
Jul 22 13:35:13.965534 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.965489    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmnp8\" (UniqueName: \"kubernetes.io/projected/3b95fd63-c6fa-4577-9e71-d50283f33117-kube-api-access-zmnp8\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:13.965743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.965550    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b95fd63-c6fa-4577-9e71-d50283f33117-openshift-service-ca\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:13.965743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:13.965644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:14.066677 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:14.066633    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:14.066857 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:14.066737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmnp8\" (UniqueName: \"kubernetes.io/projected/3b95fd63-c6fa-4577-9e71-d50283f33117-kube-api-access-zmnp8\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:14.066857 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:35:14.066779    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-serving-cert: secret "maas-api-e2e-shared-b-19e216-serving-cert" not found
Jul 22 13:35:14.066857 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:14.066802    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b95fd63-c6fa-4577-9e71-d50283f33117-openshift-service-ca\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:14.066857 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:35:14.066845    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls podName:3b95fd63-c6fa-4577-9e71-d50283f33117 nodeName:}" failed. No retries permitted until 2026-07-22 13:35:14.566828866 +0000 UTC m=+2466.858809878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls") pod "maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5" (UID: "3b95fd63-c6fa-4577-9e71-d50283f33117") : secret "maas-api-e2e-shared-b-19e216-serving-cert" not found
Jul 22 13:35:14.067636 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:14.067615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b95fd63-c6fa-4577-9e71-d50283f33117-openshift-service-ca\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:14.092156 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:14.092123    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmnp8\" (UniqueName: \"kubernetes.io/projected/3b95fd63-c6fa-4577-9e71-d50283f33117-kube-api-access-zmnp8\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:14.573540 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:14.573501    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:14.573706 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:35:14.573673    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-serving-cert: secret "maas-api-e2e-shared-b-19e216-serving-cert" not found
Jul 22 13:35:14.573756 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:35:14.573739    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls podName:3b95fd63-c6fa-4577-9e71-d50283f33117 nodeName:}" failed. No retries permitted until 2026-07-22 13:35:15.573724119 +0000 UTC m=+2467.865705130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls") pod "maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5" (UID: "3b95fd63-c6fa-4577-9e71-d50283f33117") : secret "maas-api-e2e-shared-b-19e216-serving-cert" not found
Jul 22 13:35:14.743789 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:14.743750    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:35:15.582815 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:15.582775    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:15.585322 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:15.585288    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls\") pod \"maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:15.692736 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:15.692710    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:15.836152 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:15.836124    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"]
Jul 22 13:35:15.837668 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:35:15.837640    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b95fd63_c6fa_4577_9e71_d50283f33117.slice/crio-44b58529c70f0c2103af29a43ed522a26488bf40a14ec336829e9941ecb2b1ed WatchSource:0}: Error finding container 44b58529c70f0c2103af29a43ed522a26488bf40a14ec336829e9941ecb2b1ed: Status 404 returned error can't find the container with id 44b58529c70f0c2103af29a43ed522a26488bf40a14ec336829e9941ecb2b1ed
Jul 22 13:35:16.777420 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:16.777379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5" event={"ID":"3b95fd63-c6fa-4577-9e71-d50283f33117","Type":"ContainerStarted","Data":"7e606db8310cdba3cf459f9b153468ccaf550a62e42d0da7cffbb0ed21b5dc8c"}
Jul 22 13:35:16.777819 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:16.777444    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5" event={"ID":"3b95fd63-c6fa-4577-9e71-d50283f33117","Type":"ContainerStarted","Data":"44b58529c70f0c2103af29a43ed522a26488bf40a14ec336829e9941ecb2b1ed"}
Jul 22 13:35:16.777819 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:16.777517    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:16.802978 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:16.802906    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5" podStartSLOduration=3.369504275 podStartE2EDuration="3.802891609s" totalImagesPullingTime="433.387334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:35:16.801243136 +0000 UTC m=+2469.093224170" watchObservedRunningTime="2026-07-22 13:35:16.802891609 +0000 UTC m=+2469.094872642"
Jul 22 13:35:18.789028 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:18.788898    2575 generic.go:356] "Generic (PLEG): container finished" podID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerID="a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d" exitCode=0
Jul 22 13:35:18.789028 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:18.788991    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" event={"ID":"18f7028c-4287-4e1b-a47b-816fa2d9462e","Type":"ContainerDied","Data":"a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d"}
Jul 22 13:35:18.789459 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:18.789388    2575 scope.go:117] "RemoveContainer" containerID="a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d"
Jul 22 13:35:18.790520 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:18.790493    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:18.790594 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:18.790578    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:18.790640 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:18.790626    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:19.697427 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:19.697387    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:19.796277 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:19.796242    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" event={"ID":"18f7028c-4287-4e1b-a47b-816fa2d9462e","Type":"ContainerStarted","Data":"814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9"}
Jul 22 13:35:20.266897 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:20.266862    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:20.267748 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:20.267725    2575 patch_prober.go:28] interesting pod/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 22 13:35:20.267869 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:20.267773    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 22 13:35:21.267401 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:21.267361    2575 patch_prober.go:28] interesting pod/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 22 13:35:21.268049 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:21.268016    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 22 13:35:22.267048 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:22.266570    2575 patch_prober.go:28] interesting pod/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 22 13:35:22.267048 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:22.266634    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 22 13:35:22.789197 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:22.789172    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:35:23.267213 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:23.267165    2575 patch_prober.go:28] interesting pod/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 22 13:35:23.267446 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:23.267237    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 22 13:35:24.270638 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:24.270606    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:24.271043 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:24.270865    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:24.271537 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:24.271518    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:35:27.898699 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:27.898662    2575 patch_prober.go:28] interesting pod/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": read tcp 10.133.0.2:54700->10.133.0.65:15021: read: connection reset by peer" start-of-body=
Jul 22 13:35:27.899270 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:27.899032    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": read tcp 10.133.0.2:54700->10.133.0.65:15021: read: connection reset by peer"
Jul 22 13:35:28.844911 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:28.844879    2575 generic.go:356] "Generic (PLEG): container finished" podID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerID="b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0" exitCode=0
Jul 22 13:35:28.845115 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:28.844969    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" event={"ID":"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5","Type":"ContainerDied","Data":"b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0"}
Jul 22 13:35:28.845365 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:28.845352    2575 scope.go:117] "RemoveContainer" containerID="b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0"
Jul 22 13:35:28.846449 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:28.846422    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:28.846559 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:28.846507    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:28.846559 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:28.846549    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 13:35:29.854076 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:29.854039    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" event={"ID":"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5","Type":"ContainerStarted","Data":"17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52"}
Jul 22 13:35:30.200888 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:30.200785    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:30.201089 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:30.201059    2575 patch_prober.go:28] interesting pod/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 22 13:35:30.201148 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:30.201109    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 22 13:35:31.200594 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:31.200550    2575 patch_prober.go:28] interesting pod/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 22 13:35:31.201105 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:31.200629    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 22 13:35:32.200963 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:32.200907    2575 patch_prober.go:28] interesting pod/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 22 13:35:32.201435 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:32.200988    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 22 13:35:33.200150 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:33.200113    2575 patch_prober.go:28] interesting pod/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 22 13:35:33.200326 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:33.200171    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 22 13:35:34.204290 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:34.204262    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:34.204724 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:34.204596    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:35:34.205366 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:35:34.205351    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:38:00.159307 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.159269    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"]
Jul 22 13:38:00.162966 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.162946    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.166871 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.166846    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-15ce38\"/\"kube-root-ca.crt\""
Jul 22 13:38:00.167112 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.166896    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-15ce38\"/\"openshift-service-ca.crt\""
Jul 22 13:38:00.167112 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.166901    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-15ce38\"/\"default-dockercfg-9hckg\""
Jul 22 13:38:00.167112 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.166973    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-15ce38\"/\"sub-test-model-15ce38-kserve-self-signed-certs\""
Jul 22 13:38:00.177505 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.177475    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"]
Jul 22 13:38:00.272494 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.272457    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-model-cache\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.272683 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.272504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-tmp-dir\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.272683 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.272587    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-home\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.272683 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.272655    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d45b85bf-1edc-4b4e-9788-9c1559b85285-tls-certs\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.272827 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.272695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-dshm\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.272827 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.272721    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqp8b\" (UniqueName: \"kubernetes.io/projected/d45b85bf-1edc-4b4e-9788-9c1559b85285-kube-api-access-sqp8b\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.373826 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.373791    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d45b85bf-1edc-4b4e-9788-9c1559b85285-tls-certs\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.374071 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.373857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-dshm\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.374071 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.374012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqp8b\" (UniqueName: \"kubernetes.io/projected/d45b85bf-1edc-4b4e-9788-9c1559b85285-kube-api-access-sqp8b\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.374071 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.374057    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-model-cache\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.374261 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.374090    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-tmp-dir\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.374261 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.374183    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-home\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.374504 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.374481    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-model-cache\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.374594 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.374503    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-tmp-dir\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.374594 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.374530    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-home\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.376212 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.376182    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-dshm\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.376345 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.376296    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d45b85bf-1edc-4b4e-9788-9c1559b85285-tls-certs\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.384502 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.384480    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqp8b\" (UniqueName: \"kubernetes.io/projected/d45b85bf-1edc-4b4e-9788-9c1559b85285-kube-api-access-sqp8b\") pod \"sub-test-model-15ce38-kserve-645cf44645-kf2kb\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") " pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.473071 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.472977    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:00.621041 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:00.621005    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"]
Jul 22 13:38:00.624504 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:38:00.624478    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd45b85bf_1edc_4b4e_9788_9c1559b85285.slice/crio-10acd1b3bbdf76ddeec081d8661129a38e8af9177fcce47b6d665adc297ec699 WatchSource:0}: Error finding container 10acd1b3bbdf76ddeec081d8661129a38e8af9177fcce47b6d665adc297ec699: Status 404 returned error can't find the container with id 10acd1b3bbdf76ddeec081d8661129a38e8af9177fcce47b6d665adc297ec699
Jul 22 13:38:01.590449 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:01.590399    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb" event={"ID":"d45b85bf-1edc-4b4e-9788-9c1559b85285","Type":"ContainerStarted","Data":"47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0"}
Jul 22 13:38:01.590964 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:01.590461    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb" event={"ID":"d45b85bf-1edc-4b4e-9788-9c1559b85285","Type":"ContainerStarted","Data":"10acd1b3bbdf76ddeec081d8661129a38e8af9177fcce47b6d665adc297ec699"}
Jul 22 13:38:01.620498 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:01.620441    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb" podStartSLOduration=1.620425805 podStartE2EDuration="1.620425805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:38:01.61832112 +0000 UTC m=+2633.910302154" watchObservedRunningTime="2026-07-22 13:38:01.620425805 +0000 UTC m=+2633.912406835"
Jul 22 13:38:10.473598 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:10.473560    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:10.474036 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:10.473628    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:10.486354 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:10.486322    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:38:10.648141 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:38:10.648113    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:39:09.029642 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:09.029558    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:39:09.039269 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:09.039246    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8g9v_0a9ea2b6-2062-4f0a-aa47-63859b3b2efd/ovn-acl-logging/0.log"
Jul 22 13:39:33.014414 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:33.014379    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"]
Jul 22 13:39:48.035108 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.035056    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb" podUID="d45b85bf-1edc-4b4e-9788-9c1559b85285" containerName="main" containerID="cri-o://47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0" gracePeriod=45
Jul 22 13:39:48.288982 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.288895    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:39:48.412115 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.412080    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqp8b\" (UniqueName: \"kubernetes.io/projected/d45b85bf-1edc-4b4e-9788-9c1559b85285-kube-api-access-sqp8b\") pod \"d45b85bf-1edc-4b4e-9788-9c1559b85285\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") "
Jul 22 13:39:48.412306 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.412130    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d45b85bf-1edc-4b4e-9788-9c1559b85285-tls-certs\") pod \"d45b85bf-1edc-4b4e-9788-9c1559b85285\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") "
Jul 22 13:39:48.412306 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.412205    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-home\") pod \"d45b85bf-1edc-4b4e-9788-9c1559b85285\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") "
Jul 22 13:39:48.412306 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.412238    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-tmp-dir\") pod \"d45b85bf-1edc-4b4e-9788-9c1559b85285\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") "
Jul 22 13:39:48.412306 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.412262    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-model-cache\") pod \"d45b85bf-1edc-4b4e-9788-9c1559b85285\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") "
Jul 22 13:39:48.412306 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.412300    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-dshm\") pod \"d45b85bf-1edc-4b4e-9788-9c1559b85285\" (UID: \"d45b85bf-1edc-4b4e-9788-9c1559b85285\") "
Jul 22 13:39:48.412589 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.412532    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-home" (OuterVolumeSpecName: "home") pod "d45b85bf-1edc-4b4e-9788-9c1559b85285" (UID: "d45b85bf-1edc-4b4e-9788-9c1559b85285"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:39:48.412589 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.412559    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d45b85bf-1edc-4b4e-9788-9c1559b85285" (UID: "d45b85bf-1edc-4b4e-9788-9c1559b85285"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:39:48.412675 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.412588    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-model-cache" (OuterVolumeSpecName: "model-cache") pod "d45b85bf-1edc-4b4e-9788-9c1559b85285" (UID: "d45b85bf-1edc-4b4e-9788-9c1559b85285"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:39:48.414441 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.414417    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-dshm" (OuterVolumeSpecName: "dshm") pod "d45b85bf-1edc-4b4e-9788-9c1559b85285" (UID: "d45b85bf-1edc-4b4e-9788-9c1559b85285"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:39:48.414544 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.414468    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d45b85bf-1edc-4b4e-9788-9c1559b85285-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d45b85bf-1edc-4b4e-9788-9c1559b85285" (UID: "d45b85bf-1edc-4b4e-9788-9c1559b85285"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:39:48.414544 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.414479    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d45b85bf-1edc-4b4e-9788-9c1559b85285-kube-api-access-sqp8b" (OuterVolumeSpecName: "kube-api-access-sqp8b") pod "d45b85bf-1edc-4b4e-9788-9c1559b85285" (UID: "d45b85bf-1edc-4b4e-9788-9c1559b85285"). InnerVolumeSpecName "kube-api-access-sqp8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:39:48.513402 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.513367    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-home\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:39:48.513402 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.513395    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-tmp-dir\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:39:48.513402 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.513407    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-model-cache\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:39:48.513402 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.513415    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d45b85bf-1edc-4b4e-9788-9c1559b85285-dshm\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:39:48.513682 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.513423    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqp8b\" (UniqueName: \"kubernetes.io/projected/d45b85bf-1edc-4b4e-9788-9c1559b85285-kube-api-access-sqp8b\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:39:48.513682 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:48.513432    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d45b85bf-1edc-4b4e-9788-9c1559b85285-tls-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:39:49.140153 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:49.140119    2575 generic.go:356] "Generic (PLEG): container finished" podID="d45b85bf-1edc-4b4e-9788-9c1559b85285" containerID="47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0" exitCode=0
Jul 22 13:39:49.140153 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:49.140157    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"
Jul 22 13:39:49.140657 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:49.140210    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb" event={"ID":"d45b85bf-1edc-4b4e-9788-9c1559b85285","Type":"ContainerDied","Data":"47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0"}
Jul 22 13:39:49.140657 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:49.140246    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb" event={"ID":"d45b85bf-1edc-4b4e-9788-9c1559b85285","Type":"ContainerDied","Data":"10acd1b3bbdf76ddeec081d8661129a38e8af9177fcce47b6d665adc297ec699"}
Jul 22 13:39:49.140657 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:49.140265    2575 scope.go:117] "RemoveContainer" containerID="47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0"
Jul 22 13:39:49.149312 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:49.149295    2575 scope.go:117] "RemoveContainer" containerID="47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0"
Jul 22 13:39:49.149533 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:39:49.149515    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0\": container with ID starting with 47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0 not found: ID does not exist" containerID="47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0"
Jul 22 13:39:49.149594 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:49.149541    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0"} err="failed to get container status \"47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0\": rpc error: code = NotFound desc = could not find container \"47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0\": container with ID starting with 47777c39958e568202afbc5400e513579723e5f79dd23734cfb06cbed41a76b0 not found: ID does not exist"
Jul 22 13:39:49.159406 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:49.159374    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"]
Jul 22 13:39:49.161544 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:49.161521    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-15ce38/sub-test-model-15ce38-kserve-645cf44645-kf2kb"]
Jul 22 13:39:50.948267 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:39:50.948223    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d45b85bf-1edc-4b4e-9788-9c1559b85285" path="/var/lib/kubelet/pods/d45b85bf-1edc-4b4e-9788-9c1559b85285/volumes"
Jul 22 13:40:53.258978 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.258859    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7509-139621252668080"]
Jul 22 13:40:53.261554 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.259447    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d45b85bf-1edc-4b4e-9788-9c1559b85285" containerName="main"
Jul 22 13:40:53.261554 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.259464    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d45b85bf-1edc-4b4e-9788-9c1559b85285" containerName="main"
Jul 22 13:40:53.261554 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.259532    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d45b85bf-1edc-4b4e-9788-9c1559b85285" containerName="main"
Jul 22 13:40:53.262479 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.262459    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252668080"
Jul 22 13:40:53.265052 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.265032    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4ljcl\""
Jul 22 13:40:53.272274 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.272253    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139621252668080"]
Jul 22 13:40:53.337295 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.337258    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z46bv\" (UniqueName: \"kubernetes.io/projected/0b646dff-9c3d-4379-ab54-b510faa11676-kube-api-access-z46bv\") pod \"test-curl-7509-139621252668080\" (UID: \"0b646dff-9c3d-4379-ab54-b510faa11676\") " pod="opendatahub/test-curl-7509-139621252668080"
Jul 22 13:40:53.438035 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.437997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z46bv\" (UniqueName: \"kubernetes.io/projected/0b646dff-9c3d-4379-ab54-b510faa11676-kube-api-access-z46bv\") pod \"test-curl-7509-139621252668080\" (UID: \"0b646dff-9c3d-4379-ab54-b510faa11676\") " pod="opendatahub/test-curl-7509-139621252668080"
Jul 22 13:40:53.449857 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.449828    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z46bv\" (UniqueName: \"kubernetes.io/projected/0b646dff-9c3d-4379-ab54-b510faa11676-kube-api-access-z46bv\") pod \"test-curl-7509-139621252668080\" (UID: \"0b646dff-9c3d-4379-ab54-b510faa11676\") " pod="opendatahub/test-curl-7509-139621252668080"
Jul 22 13:40:53.572581 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.572552    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252668080"
Jul 22 13:40:53.711358 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.711336    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139621252668080"]
Jul 22 13:40:53.713566 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:40:53.713538    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b646dff_9c3d_4379_ab54_b510faa11676.slice/crio-58b297e32e8d70303365d9654e1f042e0e4821afd3ea37e5cf696c55cfe1a899 WatchSource:0}: Error finding container 58b297e32e8d70303365d9654e1f042e0e4821afd3ea37e5cf696c55cfe1a899: Status 404 returned error can't find the container with id 58b297e32e8d70303365d9654e1f042e0e4821afd3ea37e5cf696c55cfe1a899
Jul 22 13:40:53.715470 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:53.715445    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 13:40:54.464762 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:54.464726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139621252668080" event={"ID":"0b646dff-9c3d-4379-ab54-b510faa11676","Type":"ContainerStarted","Data":"58b297e32e8d70303365d9654e1f042e0e4821afd3ea37e5cf696c55cfe1a899"}
Jul 22 13:40:55.471558 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:55.471518    2575 generic.go:356] "Generic (PLEG): container finished" podID="0b646dff-9c3d-4379-ab54-b510faa11676" containerID="b4eb203a4d20779e4f31787db7ac466e28a5f3b532a50167bf8efc95f89f7c99" exitCode=0
Jul 22 13:40:55.471997 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:55.471601    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139621252668080" event={"ID":"0b646dff-9c3d-4379-ab54-b510faa11676","Type":"ContainerDied","Data":"b4eb203a4d20779e4f31787db7ac466e28a5f3b532a50167bf8efc95f89f7c99"}
Jul 22 13:40:56.621957 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:56.621912    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252668080"
Jul 22 13:40:56.664265 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:56.664234    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7509-139621252668080_0b646dff-9c3d-4379-ab54-b510faa11676/test-curl-7509-139621252668080/0.log"
Jul 22 13:40:56.670410 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:56.670386    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z46bv\" (UniqueName: \"kubernetes.io/projected/0b646dff-9c3d-4379-ab54-b510faa11676-kube-api-access-z46bv\") pod \"0b646dff-9c3d-4379-ab54-b510faa11676\" (UID: \"0b646dff-9c3d-4379-ab54-b510faa11676\") "
Jul 22 13:40:56.672489 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:56.672467    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b646dff-9c3d-4379-ab54-b510faa11676-kube-api-access-z46bv" (OuterVolumeSpecName: "kube-api-access-z46bv") pod "0b646dff-9c3d-4379-ab54-b510faa11676" (UID: "0b646dff-9c3d-4379-ab54-b510faa11676"). InnerVolumeSpecName "kube-api-access-z46bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:40:56.692976 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:56.692914    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7509-139621252668080"]
Jul 22 13:40:56.702243 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:56.702210    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7509-139621252668080"]
Jul 22 13:40:56.771489 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:56.771399    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z46bv\" (UniqueName: \"kubernetes.io/projected/0b646dff-9c3d-4379-ab54-b510faa11676-kube-api-access-z46bv\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:40:56.948319 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:56.948286    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b646dff-9c3d-4379-ab54-b510faa11676" path="/var/lib/kubelet/pods/0b646dff-9c3d-4379-ab54-b510faa11676/volumes"
Jul 22 13:40:57.482511 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:57.482483    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252668080"
Jul 22 13:40:57.482511 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:57.482483    2575 scope.go:117] "RemoveContainer" containerID="b4eb203a4d20779e4f31787db7ac466e28a5f3b532a50167bf8efc95f89f7c99"
Jul 22 13:40:59.298054 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.297977    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7509-139621252580384"]
Jul 22 13:40:59.298412 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.298366    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b646dff-9c3d-4379-ab54-b510faa11676" containerName="test-curl-7509-139621252668080"
Jul 22 13:40:59.298412 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.298378    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b646dff-9c3d-4379-ab54-b510faa11676" containerName="test-curl-7509-139621252668080"
Jul 22 13:40:59.298482 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.298432    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b646dff-9c3d-4379-ab54-b510faa11676" containerName="test-curl-7509-139621252668080"
Jul 22 13:40:59.302714 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.302698    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252580384"
Jul 22 13:40:59.305279 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.305254    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4ljcl\""
Jul 22 13:40:59.306297 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.306267    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139621252580384"]
Jul 22 13:40:59.396848 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.396817    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh6nw\" (UniqueName: \"kubernetes.io/projected/2b60fd11-6522-4e07-87e6-c068d0aa3d39-kube-api-access-lh6nw\") pod \"test-curl-7509-139621252580384\" (UID: \"2b60fd11-6522-4e07-87e6-c068d0aa3d39\") " pod="opendatahub/test-curl-7509-139621252580384"
Jul 22 13:40:59.498031 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.497999    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh6nw\" (UniqueName: \"kubernetes.io/projected/2b60fd11-6522-4e07-87e6-c068d0aa3d39-kube-api-access-lh6nw\") pod \"test-curl-7509-139621252580384\" (UID: \"2b60fd11-6522-4e07-87e6-c068d0aa3d39\") " pod="opendatahub/test-curl-7509-139621252580384"
Jul 22 13:40:59.507311 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.507289    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh6nw\" (UniqueName: \"kubernetes.io/projected/2b60fd11-6522-4e07-87e6-c068d0aa3d39-kube-api-access-lh6nw\") pod \"test-curl-7509-139621252580384\" (UID: \"2b60fd11-6522-4e07-87e6-c068d0aa3d39\") " pod="opendatahub/test-curl-7509-139621252580384"
Jul 22 13:40:59.612900 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.612872    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252580384"
Jul 22 13:40:59.747334 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:40:59.747308    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139621252580384"]
Jul 22 13:40:59.749415 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:40:59.749387    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b60fd11_6522_4e07_87e6_c068d0aa3d39.slice/crio-952541f1f7ee4f972969d25b288efba13226e83d8bc3faab25f19739398bfcb0 WatchSource:0}: Error finding container 952541f1f7ee4f972969d25b288efba13226e83d8bc3faab25f19739398bfcb0: Status 404 returned error can't find the container with id 952541f1f7ee4f972969d25b288efba13226e83d8bc3faab25f19739398bfcb0
Jul 22 13:41:00.499201 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:00.499165    2575 generic.go:356] "Generic (PLEG): container finished" podID="2b60fd11-6522-4e07-87e6-c068d0aa3d39" containerID="c2a6c528338cdf50d7327bce1dc70544458e08aae82f84bf0cd2619eec142f87" exitCode=0
Jul 22 13:41:00.499588 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:00.499253    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139621252580384" event={"ID":"2b60fd11-6522-4e07-87e6-c068d0aa3d39","Type":"ContainerDied","Data":"c2a6c528338cdf50d7327bce1dc70544458e08aae82f84bf0cd2619eec142f87"}
Jul 22 13:41:00.499588 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:00.499298    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139621252580384" event={"ID":"2b60fd11-6522-4e07-87e6-c068d0aa3d39","Type":"ContainerStarted","Data":"952541f1f7ee4f972969d25b288efba13226e83d8bc3faab25f19739398bfcb0"}
Jul 22 13:41:01.640188 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.640157    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252580384"
Jul 22 13:41:01.662507 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.662470    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7509-139621252580384_2b60fd11-6522-4e07-87e6-c068d0aa3d39/test-curl-7509-139621252580384/0.log"
Jul 22 13:41:01.689654 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.689618    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7509-139621252580384"]
Jul 22 13:41:01.694108 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.694081    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7509-139621252580384"]
Jul 22 13:41:01.719780 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.719743    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh6nw\" (UniqueName: \"kubernetes.io/projected/2b60fd11-6522-4e07-87e6-c068d0aa3d39-kube-api-access-lh6nw\") pod \"2b60fd11-6522-4e07-87e6-c068d0aa3d39\" (UID: \"2b60fd11-6522-4e07-87e6-c068d0aa3d39\") "
Jul 22 13:41:01.721985 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.721954    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b60fd11-6522-4e07-87e6-c068d0aa3d39-kube-api-access-lh6nw" (OuterVolumeSpecName: "kube-api-access-lh6nw") pod "2b60fd11-6522-4e07-87e6-c068d0aa3d39" (UID: "2b60fd11-6522-4e07-87e6-c068d0aa3d39"). InnerVolumeSpecName "kube-api-access-lh6nw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:41:01.817736 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.817646    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7509-139621252581392"]
Jul 22 13:41:01.818546 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.818518    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b60fd11-6522-4e07-87e6-c068d0aa3d39" containerName="test-curl-7509-139621252580384"
Jul 22 13:41:01.818546 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.818547    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b60fd11-6522-4e07-87e6-c068d0aa3d39" containerName="test-curl-7509-139621252580384"
Jul 22 13:41:01.818910 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.818895    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b60fd11-6522-4e07-87e6-c068d0aa3d39" containerName="test-curl-7509-139621252580384"
Jul 22 13:41:01.821781 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.821749    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh6nw\" (UniqueName: \"kubernetes.io/projected/2b60fd11-6522-4e07-87e6-c068d0aa3d39-kube-api-access-lh6nw\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:01.827275 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.827242    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252581392"
Jul 22 13:41:01.828679 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.828656    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139621252581392"]
Jul 22 13:41:01.922987 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:01.922919    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w99z\" (UniqueName: \"kubernetes.io/projected/40499125-c44c-46a6-9601-00f3d90e30dc-kube-api-access-7w99z\") pod \"test-curl-7509-139621252581392\" (UID: \"40499125-c44c-46a6-9601-00f3d90e30dc\") " pod="opendatahub/test-curl-7509-139621252581392"
Jul 22 13:41:02.024214 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:02.024169    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w99z\" (UniqueName: \"kubernetes.io/projected/40499125-c44c-46a6-9601-00f3d90e30dc-kube-api-access-7w99z\") pod \"test-curl-7509-139621252581392\" (UID: \"40499125-c44c-46a6-9601-00f3d90e30dc\") " pod="opendatahub/test-curl-7509-139621252581392"
Jul 22 13:41:02.034552 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:02.034516    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w99z\" (UniqueName: \"kubernetes.io/projected/40499125-c44c-46a6-9601-00f3d90e30dc-kube-api-access-7w99z\") pod \"test-curl-7509-139621252581392\" (UID: \"40499125-c44c-46a6-9601-00f3d90e30dc\") " pod="opendatahub/test-curl-7509-139621252581392"
Jul 22 13:41:02.137486 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:02.137449    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252581392"
Jul 22 13:41:02.289598 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:02.289567    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139621252581392"]
Jul 22 13:41:02.291475 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:41:02.291432    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40499125_c44c_46a6_9601_00f3d90e30dc.slice/crio-6496e0c82fe373cb1fb43d0d0b22cce961efdcb8035b59021c1088d542bb7b0a WatchSource:0}: Error finding container 6496e0c82fe373cb1fb43d0d0b22cce961efdcb8035b59021c1088d542bb7b0a: Status 404 returned error can't find the container with id 6496e0c82fe373cb1fb43d0d0b22cce961efdcb8035b59021c1088d542bb7b0a
Jul 22 13:41:02.510237 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:02.510145    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139621252581392" event={"ID":"40499125-c44c-46a6-9601-00f3d90e30dc","Type":"ContainerStarted","Data":"6496e0c82fe373cb1fb43d0d0b22cce961efdcb8035b59021c1088d542bb7b0a"}
Jul 22 13:41:02.511340 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:02.511318    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="952541f1f7ee4f972969d25b288efba13226e83d8bc3faab25f19739398bfcb0"
Jul 22 13:41:02.511468 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:02.511348    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252580384"
Jul 22 13:41:02.522113 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:41:02.522083    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7509-139621252580384\" is forbidden: User \"system:node:ip-10-0-140-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-133.ec2.internal' and this object" podUID="2b60fd11-6522-4e07-87e6-c068d0aa3d39" pod="opendatahub/test-curl-7509-139621252580384"
Jul 22 13:41:02.948368 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:02.948333    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b60fd11-6522-4e07-87e6-c068d0aa3d39" path="/var/lib/kubelet/pods/2b60fd11-6522-4e07-87e6-c068d0aa3d39/volumes"
Jul 22 13:41:03.202359 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:03.202275    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kube-system/konnectivity-agent-khrvp" podUID="91358d07-1210-40d9-a108-654aae4213f6" containerName="konnectivity-agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 13:41:03.202505 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:03.202413    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-khrvp"
Jul 22 13:41:03.203135 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:03.203108    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="kube-system/konnectivity-agent-khrvp" podUID="91358d07-1210-40d9-a108-654aae4213f6" containerName="konnectivity-agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 13:41:03.518328 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:03.518237    2575 generic.go:356] "Generic (PLEG): container finished" podID="40499125-c44c-46a6-9601-00f3d90e30dc" containerID="162fb41b4d1a4fa55425aff442012c7ee00c57b01885e4382e9c1b062833b453" exitCode=0
Jul 22 13:41:03.518328 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:03.518297    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139621252581392" event={"ID":"40499125-c44c-46a6-9601-00f3d90e30dc","Type":"ContainerDied","Data":"162fb41b4d1a4fa55425aff442012c7ee00c57b01885e4382e9c1b062833b453"}
Jul 22 13:41:04.661113 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:04.661086    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252581392"
Jul 22 13:41:04.693239 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:04.693206    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7509-139621252581392_40499125-c44c-46a6-9601-00f3d90e30dc/test-curl-7509-139621252581392/0.log"
Jul 22 13:41:04.717210 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:04.717174    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7509-139621252581392"]
Jul 22 13:41:04.722021 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:04.721990    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7509-139621252581392"]
Jul 22 13:41:04.752283 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:04.752249    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w99z\" (UniqueName: \"kubernetes.io/projected/40499125-c44c-46a6-9601-00f3d90e30dc-kube-api-access-7w99z\") pod \"40499125-c44c-46a6-9601-00f3d90e30dc\" (UID: \"40499125-c44c-46a6-9601-00f3d90e30dc\") "
Jul 22 13:41:04.754556 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:04.754523    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40499125-c44c-46a6-9601-00f3d90e30dc-kube-api-access-7w99z" (OuterVolumeSpecName: "kube-api-access-7w99z") pod "40499125-c44c-46a6-9601-00f3d90e30dc" (UID: "40499125-c44c-46a6-9601-00f3d90e30dc"). InnerVolumeSpecName "kube-api-access-7w99z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:41:04.853606 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:04.853573    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w99z\" (UniqueName: \"kubernetes.io/projected/40499125-c44c-46a6-9601-00f3d90e30dc-kube-api-access-7w99z\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:04.948400 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:04.948356    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40499125-c44c-46a6-9601-00f3d90e30dc" path="/var/lib/kubelet/pods/40499125-c44c-46a6-9601-00f3d90e30dc/volumes"
Jul 22 13:41:05.529515 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:05.529485    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139621252581392"
Jul 22 13:41:05.529678 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:05.529542    2575 scope.go:117] "RemoveContainer" containerID="162fb41b4d1a4fa55425aff442012c7ee00c57b01885e4382e9c1b062833b453"
Jul 22 13:41:09.257118 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:09.257090    2575 scope.go:117] "RemoveContainer" containerID="cc6babec15f25eb8e110116e6f432b4ede6a65cf91e7c27024d14315371e30c0"
Jul 22 13:41:17.948276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:17.948245    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"]
Jul 22 13:41:17.948732 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:17.948470    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75" podUID="24ca63b4-63cc-48a3-946d-d1742244bf03" containerName="maas-api" containerID="cri-o://93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470" gracePeriod=30
Jul 22 13:41:18.208596 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.208525    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:41:18.388583 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.388552    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24ca63b4-63cc-48a3-946d-d1742244bf03-openshift-service-ca\") pod \"24ca63b4-63cc-48a3-946d-d1742244bf03\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") "
Jul 22 13:41:18.388583 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.388587    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24ca63b4-63cc-48a3-946d-d1742244bf03-maas-api-tls\") pod \"24ca63b4-63cc-48a3-946d-d1742244bf03\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") "
Jul 22 13:41:18.388827 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.388614    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx5dq\" (UniqueName: \"kubernetes.io/projected/24ca63b4-63cc-48a3-946d-d1742244bf03-kube-api-access-cx5dq\") pod \"24ca63b4-63cc-48a3-946d-d1742244bf03\" (UID: \"24ca63b4-63cc-48a3-946d-d1742244bf03\") "
Jul 22 13:41:18.389039 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.389012    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24ca63b4-63cc-48a3-946d-d1742244bf03-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "24ca63b4-63cc-48a3-946d-d1742244bf03" (UID: "24ca63b4-63cc-48a3-946d-d1742244bf03"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:41:18.390782 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.390756    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ca63b4-63cc-48a3-946d-d1742244bf03-kube-api-access-cx5dq" (OuterVolumeSpecName: "kube-api-access-cx5dq") pod "24ca63b4-63cc-48a3-946d-d1742244bf03" (UID: "24ca63b4-63cc-48a3-946d-d1742244bf03"). InnerVolumeSpecName "kube-api-access-cx5dq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:41:18.390782 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.390764    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24ca63b4-63cc-48a3-946d-d1742244bf03-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "24ca63b4-63cc-48a3-946d-d1742244bf03" (UID: "24ca63b4-63cc-48a3-946d-d1742244bf03"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:41:18.489377 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.489277    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24ca63b4-63cc-48a3-946d-d1742244bf03-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:18.489377 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.489311    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24ca63b4-63cc-48a3-946d-d1742244bf03-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:18.489377 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.489324    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx5dq\" (UniqueName: \"kubernetes.io/projected/24ca63b4-63cc-48a3-946d-d1742244bf03-kube-api-access-cx5dq\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:18.593021 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.592987    2575 generic.go:356] "Generic (PLEG): container finished" podID="24ca63b4-63cc-48a3-946d-d1742244bf03" containerID="93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470" exitCode=0
Jul 22 13:41:18.593205 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.593033    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"
Jul 22 13:41:18.593205 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.593040    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75" event={"ID":"24ca63b4-63cc-48a3-946d-d1742244bf03","Type":"ContainerDied","Data":"93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470"}
Jul 22 13:41:18.593205 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.593083    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75" event={"ID":"24ca63b4-63cc-48a3-946d-d1742244bf03","Type":"ContainerDied","Data":"9b65c797760f7153c4cf9f94167069809ce63b77f7b5b7b1e7bff4f4b23f5124"}
Jul 22 13:41:18.593205 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.593105    2575 scope.go:117] "RemoveContainer" containerID="93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470"
Jul 22 13:41:18.602781 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.602763    2575 scope.go:117] "RemoveContainer" containerID="93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470"
Jul 22 13:41:18.603067 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:41:18.603044    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470\": container with ID starting with 93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470 not found: ID does not exist" containerID="93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470"
Jul 22 13:41:18.603134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.603078    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470"} err="failed to get container status \"93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470\": rpc error: code = NotFound desc = could not find container \"93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470\": container with ID starting with 93d96968fe6ad0f9a5f824b072a5f620ea6421dd69d3053ecd60f1d787c94470 not found: ID does not exist"
Jul 22 13:41:18.616937 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.616893    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"]
Jul 22 13:41:18.621134 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.621106    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-15ce38-6dc584795f-xdl75"]
Jul 22 13:41:18.948530 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:18.948488    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24ca63b4-63cc-48a3-946d-d1742244bf03" path="/var/lib/kubelet/pods/24ca63b4-63cc-48a3-946d-d1742244bf03/volumes"
Jul 22 13:41:29.570244 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.570211    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"]
Jul 22 13:41:29.570794 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.570441    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerName="istio-proxy" containerID="cri-o://814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9" gracePeriod=30
Jul 22 13:41:29.953127 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.953092    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"]
Jul 22 13:41:29.953600 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.953574    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40499125-c44c-46a6-9601-00f3d90e30dc" containerName="test-curl-7509-139621252581392"
Jul 22 13:41:29.953600 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.953595    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="40499125-c44c-46a6-9601-00f3d90e30dc" containerName="test-curl-7509-139621252581392"
Jul 22 13:41:29.953600 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.953604    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24ca63b4-63cc-48a3-946d-d1742244bf03" containerName="maas-api"
Jul 22 13:41:29.953819 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.953610    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="24ca63b4-63cc-48a3-946d-d1742244bf03" containerName="maas-api"
Jul 22 13:41:29.953819 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.953668    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="24ca63b4-63cc-48a3-946d-d1742244bf03" containerName="maas-api"
Jul 22 13:41:29.953819 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.953676    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="40499125-c44c-46a6-9601-00f3d90e30dc" containerName="test-curl-7509-139621252581392"
Jul 22 13:41:29.958385 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.958367    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"
Jul 22 13:41:29.961427 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.961399    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bpvdk\""
Jul 22 13:41:29.963944 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:29.963901    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"]
Jul 22 13:41:30.001681 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:30.001645    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/009e19aa-c34b-4b46-80f3-34e3d5181af2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk\" (UID: \"009e19aa-c34b-4b46-80f3-34e3d5181af2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"
Jul 22 13:41:30.103044 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:30.103002    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/009e19aa-c34b-4b46-80f3-34e3d5181af2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk\" (UID: \"009e19aa-c34b-4b46-80f3-34e3d5181af2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"
Jul 22 13:41:30.103683 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:30.103661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/009e19aa-c34b-4b46-80f3-34e3d5181af2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk\" (UID: \"009e19aa-c34b-4b46-80f3-34e3d5181af2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"
Jul 22 13:41:30.271421 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:30.271326    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"
Jul 22 13:41:30.415533 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:30.415502    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"]
Jul 22 13:41:30.416875 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:41:30.416840    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod009e19aa_c34b_4b46_80f3_34e3d5181af2.slice/crio-85b9cab1c13830a14d117aec111ae6af4f5709648fdc304d8dca04efc24d16c4 WatchSource:0}: Error finding container 85b9cab1c13830a14d117aec111ae6af4f5709648fdc304d8dca04efc24d16c4: Status 404 returned error can't find the container with id 85b9cab1c13830a14d117aec111ae6af4f5709648fdc304d8dca04efc24d16c4
Jul 22 13:41:30.657986 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:30.657957    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk" event={"ID":"009e19aa-c34b-4b46-80f3-34e3d5181af2","Type":"ContainerStarted","Data":"85b9cab1c13830a14d117aec111ae6af4f5709648fdc304d8dca04efc24d16c4"}
Jul 22 13:41:31.663778 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:31.663739    2575 generic.go:356] "Generic (PLEG): container finished" podID="009e19aa-c34b-4b46-80f3-34e3d5181af2" containerID="793903de6dbe4edfdced2e9e10d50491cc8911a3d9a670e074547090fd1f391f" exitCode=0
Jul 22 13:41:31.664183 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:31.663785    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk" event={"ID":"009e19aa-c34b-4b46-80f3-34e3d5181af2","Type":"ContainerDied","Data":"793903de6dbe4edfdced2e9e10d50491cc8911a3d9a670e074547090fd1f391f"}
Jul 22 13:41:32.805469 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:32.805444    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"
Jul 22 13:41:32.928743 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:32.928633    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/009e19aa-c34b-4b46-80f3-34e3d5181af2-maas-api-service-ca\") pod \"009e19aa-c34b-4b46-80f3-34e3d5181af2\" (UID: \"009e19aa-c34b-4b46-80f3-34e3d5181af2\") "
Jul 22 13:41:32.929040 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:32.929017    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/009e19aa-c34b-4b46-80f3-34e3d5181af2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "009e19aa-c34b-4b46-80f3-34e3d5181af2" (UID: "009e19aa-c34b-4b46-80f3-34e3d5181af2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:41:33.030302 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:33.030263    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/009e19aa-c34b-4b46-80f3-34e3d5181af2-maas-api-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:33.204384 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:33.204293    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-khrvp"
Jul 22 13:41:33.677786 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:33.677750    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"
Jul 22 13:41:33.677987 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:33.677779    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk" event={"ID":"009e19aa-c34b-4b46-80f3-34e3d5181af2","Type":"ContainerDied","Data":"85b9cab1c13830a14d117aec111ae6af4f5709648fdc304d8dca04efc24d16c4"}
Jul 22 13:41:33.677987 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:33.677838    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85b9cab1c13830a14d117aec111ae6af4f5709648fdc304d8dca04efc24d16c4"
Jul 22 13:41:34.843465 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.843403    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:41:34.945741 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.945713    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-socket\") pod \"18f7028c-4287-4e1b-a47b-816fa2d9462e\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") "
Jul 22 13:41:34.945915 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.945769    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-credential-socket\") pod \"18f7028c-4287-4e1b-a47b-816fa2d9462e\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") "
Jul 22 13:41:34.945915 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.945796    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18f7028c-4287-4e1b-a47b-816fa2d9462e-istiod-ca-cert\") pod \"18f7028c-4287-4e1b-a47b-816fa2d9462e\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") "
Jul 22 13:41:34.945915 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.945839    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-podinfo\") pod \"18f7028c-4287-4e1b-a47b-816fa2d9462e\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") "
Jul 22 13:41:34.945915 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.945881    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fb5c\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-kube-api-access-9fb5c\") pod \"18f7028c-4287-4e1b-a47b-816fa2d9462e\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") "
Jul 22 13:41:34.945915 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.945900    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-certs\") pod \"18f7028c-4287-4e1b-a47b-816fa2d9462e\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") "
Jul 22 13:41:34.946200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946044    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18f7028c-4287-4e1b-a47b-816fa2d9462e" (UID: "18f7028c-4287-4e1b-a47b-816fa2d9462e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:34.946200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946062    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-envoy\") pod \"18f7028c-4287-4e1b-a47b-816fa2d9462e\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") "
Jul 22 13:41:34.946200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946118    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18f7028c-4287-4e1b-a47b-816fa2d9462e" (UID: "18f7028c-4287-4e1b-a47b-816fa2d9462e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:34.946200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946132    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-data\") pod \"18f7028c-4287-4e1b-a47b-816fa2d9462e\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") "
Jul 22 13:41:34.946200 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946168    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-token\") pod \"18f7028c-4287-4e1b-a47b-816fa2d9462e\" (UID: \"18f7028c-4287-4e1b-a47b-816fa2d9462e\") "
Jul 22 13:41:34.946471 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946237    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18f7028c-4287-4e1b-a47b-816fa2d9462e" (UID: "18f7028c-4287-4e1b-a47b-816fa2d9462e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:34.946471 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946463    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:34.946573 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946481    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-workload-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:34.946573 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946494    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-credential-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:34.946865 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946840    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f7028c-4287-4e1b-a47b-816fa2d9462e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18f7028c-4287-4e1b-a47b-816fa2d9462e" (UID: "18f7028c-4287-4e1b-a47b-816fa2d9462e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:41:34.947046 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.946953    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-data" (OuterVolumeSpecName: "istio-data") pod "18f7028c-4287-4e1b-a47b-816fa2d9462e" (UID: "18f7028c-4287-4e1b-a47b-816fa2d9462e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:34.948458 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.948432    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18f7028c-4287-4e1b-a47b-816fa2d9462e" (UID: "18f7028c-4287-4e1b-a47b-816fa2d9462e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:41:34.948576 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.948480    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-token" (OuterVolumeSpecName: "istio-token") pod "18f7028c-4287-4e1b-a47b-816fa2d9462e" (UID: "18f7028c-4287-4e1b-a47b-816fa2d9462e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:41:34.948576 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.948529    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18f7028c-4287-4e1b-a47b-816fa2d9462e" (UID: "18f7028c-4287-4e1b-a47b-816fa2d9462e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:34.948576 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:34.948558    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-kube-api-access-9fb5c" (OuterVolumeSpecName: "kube-api-access-9fb5c") pod "18f7028c-4287-4e1b-a47b-816fa2d9462e" (UID: "18f7028c-4287-4e1b-a47b-816fa2d9462e"). InnerVolumeSpecName "kube-api-access-9fb5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:41:35.047301 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.047264    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18f7028c-4287-4e1b-a47b-816fa2d9462e-istiod-ca-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:35.047301 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.047292    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-podinfo\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:35.047301 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.047300    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fb5c\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-kube-api-access-9fb5c\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:35.047525 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.047313    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-envoy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:35.047525 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.047325    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-data\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:35.047525 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.047332    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18f7028c-4287-4e1b-a47b-816fa2d9462e-istio-token\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:35.688884 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.688854    2575 generic.go:356] "Generic (PLEG): container finished" podID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerID="814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9" exitCode=0
Jul 22 13:41:35.689081 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.688903    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"
Jul 22 13:41:35.689081 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.688944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" event={"ID":"18f7028c-4287-4e1b-a47b-816fa2d9462e","Type":"ContainerDied","Data":"814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9"}
Jul 22 13:41:35.689081 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.688992    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd" event={"ID":"18f7028c-4287-4e1b-a47b-816fa2d9462e","Type":"ContainerDied","Data":"f95248fa6f19e24f7b1de127717e00f46b891a4137eb97412a68c9ce518ff4c4"}
Jul 22 13:41:35.689081 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.689018    2575 scope.go:117] "RemoveContainer" containerID="814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9"
Jul 22 13:41:35.698658 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.698638    2575 scope.go:117] "RemoveContainer" containerID="a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d"
Jul 22 13:41:35.706419 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.706398    2575 scope.go:117] "RemoveContainer" containerID="814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9"
Jul 22 13:41:35.706779 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:41:35.706745    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9\": container with ID starting with 814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9 not found: ID does not exist" containerID="814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9"
Jul 22 13:41:35.706878 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.706789    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9"} err="failed to get container status \"814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9\": rpc error: code = NotFound desc = could not find container \"814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9\": container with ID starting with 814968ae665e0d599a0fb6761c3f9c0569ce118c781fd7e972c47999c3d576c9 not found: ID does not exist"
Jul 22 13:41:35.706878 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.706813    2575 scope.go:117] "RemoveContainer" containerID="a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d"
Jul 22 13:41:35.707328 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:41:35.707301    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d\": container with ID starting with a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d not found: ID does not exist" containerID="a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d"
Jul 22 13:41:35.707445 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.707332    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d"} err="failed to get container status \"a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d\": rpc error: code = NotFound desc = could not find container \"a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d\": container with ID starting with a1c33a5475276c6da965cbdc83dfacf4cb2aa75864166b28e5179cd4190f073d not found: ID does not exist"
Jul 22 13:41:35.709122 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.709103    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"]
Jul 22 13:41:35.713942 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:35.713911    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-15ce38-openshift-default-85bbdc4555-zdcmd"]
Jul 22 13:41:36.947706 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:36.947674    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" path="/var/lib/kubelet/pods/18f7028c-4287-4e1b-a47b-816fa2d9462e/volumes"
Jul 22 13:41:40.480850 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.480815    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"]
Jul 22 13:41:40.481291 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.481042    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5" podUID="3b95fd63-c6fa-4577-9e71-d50283f33117" containerName="maas-api" containerID="cri-o://7e606db8310cdba3cf459f9b153468ccaf550a62e42d0da7cffbb0ed21b5dc8c" gracePeriod=30
Jul 22 13:41:40.734245 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.734211    2575 generic.go:356] "Generic (PLEG): container finished" podID="3b95fd63-c6fa-4577-9e71-d50283f33117" containerID="7e606db8310cdba3cf459f9b153468ccaf550a62e42d0da7cffbb0ed21b5dc8c" exitCode=0
Jul 22 13:41:40.734373 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.734276    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5" event={"ID":"3b95fd63-c6fa-4577-9e71-d50283f33117","Type":"ContainerDied","Data":"7e606db8310cdba3cf459f9b153468ccaf550a62e42d0da7cffbb0ed21b5dc8c"}
Jul 22 13:41:40.752859 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.752833    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:41:40.901777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.901747    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls\") pod \"3b95fd63-c6fa-4577-9e71-d50283f33117\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") "
Jul 22 13:41:40.901777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.901787    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b95fd63-c6fa-4577-9e71-d50283f33117-openshift-service-ca\") pod \"3b95fd63-c6fa-4577-9e71-d50283f33117\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") "
Jul 22 13:41:40.902034 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.901818    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmnp8\" (UniqueName: \"kubernetes.io/projected/3b95fd63-c6fa-4577-9e71-d50283f33117-kube-api-access-zmnp8\") pod \"3b95fd63-c6fa-4577-9e71-d50283f33117\" (UID: \"3b95fd63-c6fa-4577-9e71-d50283f33117\") "
Jul 22 13:41:40.902232 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.902211    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b95fd63-c6fa-4577-9e71-d50283f33117-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b95fd63-c6fa-4577-9e71-d50283f33117" (UID: "3b95fd63-c6fa-4577-9e71-d50283f33117"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:41:40.904011 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.903983    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b95fd63-c6fa-4577-9e71-d50283f33117-kube-api-access-zmnp8" (OuterVolumeSpecName: "kube-api-access-zmnp8") pod "3b95fd63-c6fa-4577-9e71-d50283f33117" (UID: "3b95fd63-c6fa-4577-9e71-d50283f33117"). InnerVolumeSpecName "kube-api-access-zmnp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:41:40.904011 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:40.903991    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b95fd63-c6fa-4577-9e71-d50283f33117" (UID: "3b95fd63-c6fa-4577-9e71-d50283f33117"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:41:41.003099 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:41.003002    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b95fd63-c6fa-4577-9e71-d50283f33117-maas-api-tls\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:41.003099 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:41.003030    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b95fd63-c6fa-4577-9e71-d50283f33117-openshift-service-ca\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:41.003099 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:41.003048    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmnp8\" (UniqueName: \"kubernetes.io/projected/3b95fd63-c6fa-4577-9e71-d50283f33117-kube-api-access-zmnp8\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:41.740099 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:41.740059    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5" event={"ID":"3b95fd63-c6fa-4577-9e71-d50283f33117","Type":"ContainerDied","Data":"44b58529c70f0c2103af29a43ed522a26488bf40a14ec336829e9941ecb2b1ed"}
Jul 22 13:41:41.740502 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:41.740070    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"
Jul 22 13:41:41.740502 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:41.740115    2575 scope.go:117] "RemoveContainer" containerID="7e606db8310cdba3cf459f9b153468ccaf550a62e42d0da7cffbb0ed21b5dc8c"
Jul 22 13:41:41.760754 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:41.760728    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"]
Jul 22 13:41:41.765633 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:41.765599    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-19e216-7cd7dcf9f9-b74m5"]
Jul 22 13:41:42.948251 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:42.948217    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b95fd63-c6fa-4577-9e71-d50283f33117" path="/var/lib/kubelet/pods/3b95fd63-c6fa-4577-9e71-d50283f33117/volumes"
Jul 22 13:41:45.035788 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:45.035756    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"]
Jul 22 13:41:45.039018 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:45.038992    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-19e216-gjjfk"]
Jul 22 13:41:46.948123 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:46.948092    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="009e19aa-c34b-4b46-80f3-34e3d5181af2" path="/var/lib/kubelet/pods/009e19aa-c34b-4b46-80f3-34e3d5181af2/volumes"
Jul 22 13:41:52.205340 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:52.205308    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"]
Jul 22 13:41:52.205723 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:52.205527    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerName="istio-proxy" containerID="cri-o://17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52" gracePeriod=30
Jul 22 13:41:52.915175 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:52.915138    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bd7fb4977-n7lqs_1d2ae148-3643-41cd-a1c1-00dea6fd3a0f/authorino/0.log"
Jul 22 13:41:57.462022 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.461994    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:41:57.653500 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653458    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-token\") pod \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") "
Jul 22 13:41:57.653500 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653511    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-data\") pod \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") "
Jul 22 13:41:57.653759 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653538    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gc45\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-kube-api-access-4gc45\") pod \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") "
Jul 22 13:41:57.653759 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653569    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-socket\") pod \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") "
Jul 22 13:41:57.653759 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653593    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istiod-ca-cert\") pod \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") "
Jul 22 13:41:57.653759 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653607    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-certs\") pod \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") "
Jul 22 13:41:57.653759 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653629    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-podinfo\") pod \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") "
Jul 22 13:41:57.653759 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653689    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-envoy\") pod \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") "
Jul 22 13:41:57.653759 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653738    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-credential-socket\") pod \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\" (UID: \"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5\") "
Jul 22 13:41:57.654136 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.653876    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" (UID: "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:57.654136 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.654102    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:57.654136 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.654113    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" (UID: "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:57.654333 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.654303    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" (UID: "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:41:57.654459 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.654381    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" (UID: "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:57.654592 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.654566    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" (UID: "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:57.655830 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.655811    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" (UID: "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:41:57.655949 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.655836    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-kube-api-access-4gc45" (OuterVolumeSpecName: "kube-api-access-4gc45") pod "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" (UID: "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5"). InnerVolumeSpecName "kube-api-access-4gc45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:41:57.656259 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.656242    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" (UID: "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:41:57.656336 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.656259    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" (UID: "0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:41:57.755155 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.755122    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-token\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:57.755155 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.755149    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-data\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:57.755155 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.755160    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gc45\" (UniqueName: \"kubernetes.io/projected/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-kube-api-access-4gc45\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:57.755562 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.755169    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istiod-ca-cert\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:57.755562 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.755178    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-workload-certs\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:57.755562 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.755185    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-podinfo\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:57.755562 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.755192    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-istio-envoy\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:57.755562 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.755201    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5-credential-socket\") on node \"ip-10-0-140-133.ec2.internal\" DevicePath \"\""
Jul 22 13:41:57.816194 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.816164    2575 generic.go:356] "Generic (PLEG): container finished" podID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerID="17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52" exitCode=0
Jul 22 13:41:57.816353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.816212    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"
Jul 22 13:41:57.816353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.816233    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" event={"ID":"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5","Type":"ContainerDied","Data":"17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52"}
Jul 22 13:41:57.816353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.816276    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf" event={"ID":"0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5","Type":"ContainerDied","Data":"7539892b2786642829a114352d38cb69cdf4d9d13b8db7eecbc0c8f71e98a501"}
Jul 22 13:41:57.816353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.816297    2575 scope.go:117] "RemoveContainer" containerID="17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52"
Jul 22 13:41:57.825917 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.825901    2575 scope.go:117] "RemoveContainer" containerID="b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0"
Jul 22 13:41:57.833719 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.833702    2575 scope.go:117] "RemoveContainer" containerID="17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52"
Jul 22 13:41:57.833971 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:41:57.833951    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52\": container with ID starting with 17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52 not found: ID does not exist" containerID="17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52"
Jul 22 13:41:57.834026 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.833981    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52"} err="failed to get container status \"17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52\": rpc error: code = NotFound desc = could not find container \"17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52\": container with ID starting with 17bc1ab2d44c45d791452920da8d5e2f588a597feeb402de9b50c7fea7eded52 not found: ID does not exist"
Jul 22 13:41:57.834026 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.833998    2575 scope.go:117] "RemoveContainer" containerID="b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0"
Jul 22 13:41:57.834261 ip-10-0-140-133 kubenswrapper[2575]: E0722 13:41:57.834240    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0\": container with ID starting with b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0 not found: ID does not exist" containerID="b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0"
Jul 22 13:41:57.834301 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.834267    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0"} err="failed to get container status \"b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0\": rpc error: code = NotFound desc = could not find container \"b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0\": container with ID starting with b81ecde4cd6fca4767e8afbf72fe2fc41871ab61290acf2103316aaefc61cfc0 not found: ID does not exist"
Jul 22 13:41:57.840215 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.840189    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"]
Jul 22 13:41:57.845634 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:57.845612    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-19e216-openshift-default-5898b8dfcf-9qjhf"]
Jul 22 13:41:58.948346 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:58.948316    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" path="/var/lib/kubelet/pods/0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5/volumes"
Jul 22 13:41:59.462887 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:59.462805    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74989f8477-n87vv_d980c01e-8def-4a8e-af30-071333cea555/copy-manifests/0.log"
Jul 22 13:41:59.483597 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:59.483563    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74989f8477-n87vv_d980c01e-8def-4a8e-af30-071333cea555/manager/0.log"
Jul 22 13:41:59.966276 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:41:59.966250    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745435-ljkw6_c1904046-c101-411e-b3cf-6aa7ae16a0cc/cleanup/0.log"
Jul 22 13:42:00.086434 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:00.086405    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745450-6gcqb_4ea2e02e-00f7-425b-ac08-c188194cde01/cleanup/0.log"
Jul 22 13:42:01.575905 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:01.575862    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bd7fb4977-n7lqs_1d2ae148-3643-41cd-a1c1-00dea6fd3a0f/authorino/0.log"
Jul 22 13:42:03.072315 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:03.072285    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-767b4d54f8-5r995_fda46392-a2b7-46cc-828f-03f688ebd9ea/kube-auth-proxy/0.log"
Jul 22 13:42:04.154312 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:04.154283    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-r8c59_c1936ed5-dd0f-47a1-872b-5b624998506d/main/0.log"
Jul 22 13:42:04.273382 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:04.273354    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-xcth2_63839fd0-46f5-4abb-b225-1ffb52e19f5d/main/0.log"
Jul 22 13:42:04.517659 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:04.517580    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-px5xp_82890132-3dd0-4885-bae0-e5fb6dd7b308/main/0.log"
Jul 22 13:42:14.615455 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:14.615404    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5cxpk_2436034c-df7c-4212-b9d8-b85737f6c858/global-pull-secret-syncer/0.log"
Jul 22 13:42:14.752168 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:14.752137    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-khrvp_91358d07-1210-40d9-a108-654aae4213f6/konnectivity-agent/0.log"
Jul 22 13:42:14.856671 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:14.856639    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-133.ec2.internal_7be2de9b8daaa41c577843a39b0d55eb/haproxy/0.log"
Jul 22 13:42:19.378461 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:19.378429    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bd7fb4977-n7lqs_1d2ae148-3643-41cd-a1c1-00dea6fd3a0f/authorino/0.log"
Jul 22 13:42:21.724512 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:21.724469    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-5dpk9_8488358c-f988-4072-8d9b-be6e9f10a0ba/kube-state-metrics/0.log"
Jul 22 13:42:21.746670 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:21.746642    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-5dpk9_8488358c-f988-4072-8d9b-be6e9f10a0ba/kube-rbac-proxy-main/0.log"
Jul 22 13:42:21.783273 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:21.783251    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-5dpk9_8488358c-f988-4072-8d9b-be6e9f10a0ba/kube-rbac-proxy-self/0.log"
Jul 22 13:42:21.858369 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:21.858337    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-qvcvc_bd71ac03-0e60-4347-9e79-253b9281f35c/monitoring-plugin/0.log"
Jul 22 13:42:21.894369 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:21.894341    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8t48c_b6404337-0437-4dad-97d6-52ee8e19ca04/node-exporter/0.log"
Jul 22 13:42:21.918835 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:21.918807    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8t48c_b6404337-0437-4dad-97d6-52ee8e19ca04/kube-rbac-proxy/0.log"
Jul 22 13:42:21.941881 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:21.941858    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8t48c_b6404337-0437-4dad-97d6-52ee8e19ca04/init-textfile/0.log"
Jul 22 13:42:22.353341 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.353312    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eeb39e7e-95b3-40cc-8e3a-c81d36dc3315/prometheus/0.log"
Jul 22 13:42:22.386202 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.386117    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eeb39e7e-95b3-40cc-8e3a-c81d36dc3315/config-reloader/0.log"
Jul 22 13:42:22.413353 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.413327    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eeb39e7e-95b3-40cc-8e3a-c81d36dc3315/thanos-sidecar/0.log"
Jul 22 13:42:22.440585 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.440557    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eeb39e7e-95b3-40cc-8e3a-c81d36dc3315/kube-rbac-proxy-web/0.log"
Jul 22 13:42:22.468229 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.468205    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eeb39e7e-95b3-40cc-8e3a-c81d36dc3315/kube-rbac-proxy/0.log"
Jul 22 13:42:22.496695 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.496670    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eeb39e7e-95b3-40cc-8e3a-c81d36dc3315/kube-rbac-proxy-thanos/0.log"
Jul 22 13:42:22.524593 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.524556    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_eeb39e7e-95b3-40cc-8e3a-c81d36dc3315/init-config-reloader/0.log"
Jul 22 13:42:22.560568 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.560541    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-fxp7m_f8017fde-4dbe-4e1b-8ba6-d12d2098df9d/prometheus-operator/0.log"
Jul 22 13:42:22.581550 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.581523    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-fxp7m_f8017fde-4dbe-4e1b-8ba6-d12d2098df9d/kube-rbac-proxy/0.log"
Jul 22 13:42:22.611138 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.611113    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-wfrwl_0502af9d-e8c9-4ea7-92a1-c2d11aa10d48/prometheus-operator-admission-webhook/0.log"
Jul 22 13:42:22.767975 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.767868    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f597666cd-d9n4j_44420be8-bc55-4236-a425-eae4067ef8e3/thanos-query/0.log"
Jul 22 13:42:22.804070 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.804044    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f597666cd-d9n4j_44420be8-bc55-4236-a425-eae4067ef8e3/kube-rbac-proxy-web/0.log"
Jul 22 13:42:22.831143 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.831121    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f597666cd-d9n4j_44420be8-bc55-4236-a425-eae4067ef8e3/kube-rbac-proxy/0.log"
Jul 22 13:42:22.865756 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.865730    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f597666cd-d9n4j_44420be8-bc55-4236-a425-eae4067ef8e3/prom-label-proxy/0.log"
Jul 22 13:42:22.893986 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.893962    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f597666cd-d9n4j_44420be8-bc55-4236-a425-eae4067ef8e3/kube-rbac-proxy-rules/0.log"
Jul 22 13:42:22.928936 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:22.928906    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f597666cd-d9n4j_44420be8-bc55-4236-a425-eae4067ef8e3/kube-rbac-proxy-metrics/0.log"
Jul 22 13:42:25.362639 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:25.362604    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-rt7l9_cc75880f-e47d-40b0-aded-055b41bc1a02/download-server/0.log"
Jul 22 13:42:26.583399 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583363    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"]
Jul 22 13:42:26.583777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583733    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerName="istio-proxy"
Jul 22 13:42:26.583777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583745    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerName="istio-proxy"
Jul 22 13:42:26.583777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583754    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerName="istio-proxy"
Jul 22 13:42:26.583777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583759    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerName="istio-proxy"
Jul 22 13:42:26.583777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583766    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="009e19aa-c34b-4b46-80f3-34e3d5181af2" containerName="revoke-keys"
Jul 22 13:42:26.583777 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583771    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="009e19aa-c34b-4b46-80f3-34e3d5181af2" containerName="revoke-keys"
Jul 22 13:42:26.584022 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583785    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b95fd63-c6fa-4577-9e71-d50283f33117" containerName="maas-api"
Jul 22 13:42:26.584022 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583791    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b95fd63-c6fa-4577-9e71-d50283f33117" containerName="maas-api"
Jul 22 13:42:26.584022 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583854    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d4cb7b0-f8e2-4d87-b2e6-10b475e904d5" containerName="istio-proxy"
Jul 22 13:42:26.584022 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583861    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="009e19aa-c34b-4b46-80f3-34e3d5181af2" containerName="revoke-keys"
Jul 22 13:42:26.584022 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583869    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerName="istio-proxy"
Jul 22 13:42:26.584022 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583874    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f7028c-4287-4e1b-a47b-816fa2d9462e" containerName="istio-proxy"
Jul 22 13:42:26.584022 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.583880    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b95fd63-c6fa-4577-9e71-d50283f33117" containerName="maas-api"
Jul 22 13:42:26.586864 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.586848    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.589747 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.589731    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9zstf\"/\"openshift-service-ca.crt\""
Jul 22 13:42:26.590782 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.590762    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9zstf\"/\"default-dockercfg-bnjsf\""
Jul 22 13:42:26.590906 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.590792    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9zstf\"/\"kube-root-ca.crt\""
Jul 22 13:42:26.599490 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.599467    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"]
Jul 22 13:42:26.702584 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.702549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zctjk\" (UniqueName: \"kubernetes.io/projected/b4df2b79-d5ce-4565-aa2a-0f1f81248769-kube-api-access-zctjk\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.702584 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.702585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-podres\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.702814 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.702662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-sys\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.702814 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.702693    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-proc\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.702814 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.702727    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-lib-modules\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.804202 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.804166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-sys\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.804390 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.804213    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-proc\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.804390 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.804248    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-lib-modules\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.804390 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.804287    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zctjk\" (UniqueName: \"kubernetes.io/projected/b4df2b79-d5ce-4565-aa2a-0f1f81248769-kube-api-access-zctjk\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.804390 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.804293    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-sys\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.804390 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.804316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-podres\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.804390 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.804363    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-proc\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.804598 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.804439    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-lib-modules\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.804598 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.804476    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b4df2b79-d5ce-4565-aa2a-0f1f81248769-podres\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.814423 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.814400    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zctjk\" (UniqueName: \"kubernetes.io/projected/b4df2b79-d5ce-4565-aa2a-0f1f81248769-kube-api-access-zctjk\") pod \"perf-node-gather-daemonset-vqhdm\" (UID: \"b4df2b79-d5ce-4565-aa2a-0f1f81248769\") " pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:26.896270 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:26.896175    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:27.033027 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:27.033002    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"]
Jul 22 13:42:27.038267 ip-10-0-140-133 kubenswrapper[2575]: W0722 13:42:27.038051    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb4df2b79_d5ce_4565_aa2a_0f1f81248769.slice/crio-c61a7131e1bc07b6af09127529fc989abb67d045925eb1f2a328d7220da5bc1d WatchSource:0}: Error finding container c61a7131e1bc07b6af09127529fc989abb67d045925eb1f2a328d7220da5bc1d: Status 404 returned error can't find the container with id c61a7131e1bc07b6af09127529fc989abb67d045925eb1f2a328d7220da5bc1d
Jul 22 13:42:27.325565 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:27.325530    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9p9bf_e5d9191b-4abb-4410-a96c-a2665c513b75/dns/0.log"
Jul 22 13:42:27.353467 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:27.353440    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9p9bf_e5d9191b-4abb-4410-a96c-a2665c513b75/kube-rbac-proxy/0.log"
Jul 22 13:42:27.520244 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:27.520211    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qsx49_bfb3c025-7b4e-45c8-aced-0602196c44f4/dns-node-resolver/0.log"
Jul 22 13:42:27.960127 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:27.960093    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm" event={"ID":"b4df2b79-d5ce-4565-aa2a-0f1f81248769","Type":"ContainerStarted","Data":"09d4839a6afb63ee1fa265d76659e9a10b184b6bba0446ee29db6a6fe71924aa"}
Jul 22 13:42:27.960567 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:27.960150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm" event={"ID":"b4df2b79-d5ce-4565-aa2a-0f1f81248769","Type":"ContainerStarted","Data":"c61a7131e1bc07b6af09127529fc989abb67d045925eb1f2a328d7220da5bc1d"}
Jul 22 13:42:27.960567 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:27.960192    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:27.981032 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:27.980978    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm" podStartSLOduration=1.980960625 podStartE2EDuration="1.980960625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 13:42:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:42:27.978676705 +0000 UTC m=+2900.270657738" watchObservedRunningTime="2026-07-22 13:42:27.980960625 +0000 UTC m=+2900.272941658"
Jul 22 13:42:28.125161 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:28.125134    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d49b55997-brbvr_51c80405-cd6f-43de-89ad-120b15b6b0ce/registry/0.log"
Jul 22 13:42:28.184969 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:28.184916    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nsbth_05bffc4a-a65c-41b9-9bd9-7d5bf5f5d055/node-ca/0.log"
Jul 22 13:42:29.425561 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:29.425535    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-767b4d54f8-5r995_fda46392-a2b7-46cc-828f-03f688ebd9ea/kube-auth-proxy/0.log"
Jul 22 13:42:30.442771 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:30.442742    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v6nbv_1dd6eef4-46dd-469e-b5e0-49f2b5cc581b/serve-healthcheck-canary/0.log"
Jul 22 13:42:31.075050 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:31.075022    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dh668_6ae71a63-bc40-4528-8ce5-626c8d3daf02/kube-rbac-proxy/0.log"
Jul 22 13:42:31.098587 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:31.098558    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dh668_6ae71a63-bc40-4528-8ce5-626c8d3daf02/exporter/0.log"
Jul 22 13:42:31.124211 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:31.124179    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dh668_6ae71a63-bc40-4528-8ce5-626c8d3daf02/extractor/0.log"
Jul 22 13:42:33.977184 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:33.977155    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9zstf/perf-node-gather-daemonset-vqhdm"
Jul 22 13:42:34.053296 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:34.053264    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74989f8477-n87vv_d980c01e-8def-4a8e-af30-071333cea555/manager/0.log"
Jul 22 13:42:34.074638 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:34.074614    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74989f8477-n87vv_d980c01e-8def-4a8e-af30-071333cea555/copy-manifests/0.log"
Jul 22 13:42:34.817561 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:34.817533    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-767b4d54f8-5r995_fda46392-a2b7-46cc-828f-03f688ebd9ea/kube-auth-proxy/0.log"
Jul 22 13:42:35.462213 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:35.462167    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-68759d94bb-vjdjn_71770cb3-0614-4602-aa05-300f5419bd28/manager/0.log"
Jul 22 13:42:36.465501 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:36.465467    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-767b4d54f8-5r995_fda46392-a2b7-46cc-828f-03f688ebd9ea/kube-auth-proxy/0.log"
Jul 22 13:42:40.917072 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:40.917038    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-2vtp9_3b2461d0-33da-4111-a51f-be731413653a/migrator/0.log"
Jul 22 13:42:40.947405 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:40.947379    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-2vtp9_3b2461d0-33da-4111-a51f-be731413653a/graceful-termination/0.log"
Jul 22 13:42:42.806773 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:42.806742    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjtlj_8946923f-734c-497c-a2ee-73f244a994ee/kube-multus-additional-cni-plugins/0.log"
Jul 22 13:42:42.835263 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:42.835231    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjtlj_8946923f-734c-497c-a2ee-73f244a994ee/egress-router-binary-copy/0.log"
Jul 22 13:42:42.876116 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:42.876080    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjtlj_8946923f-734c-497c-a2ee-73f244a994ee/cni-plugins/0.log"
Jul 22 13:42:42.909281 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:42.909259    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjtlj_8946923f-734c-497c-a2ee-73f244a994ee/bond-cni-plugin/0.log"
Jul 22 13:42:42.932789 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:42.932716    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjtlj_8946923f-734c-497c-a2ee-73f244a994ee/routeoverride-cni/0.log"
Jul 22 13:42:42.956769 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:42.956744    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjtlj_8946923f-734c-497c-a2ee-73f244a994ee/whereabouts-cni-bincopy/0.log"
Jul 22 13:42:42.979285 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:42.979252    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjtlj_8946923f-734c-497c-a2ee-73f244a994ee/whereabouts-cni/0.log"
Jul 22 13:42:43.026327 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:43.026289    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pz4kt_0ba7a0b4-cc88-4042-beb2-3c6be9e63a20/kube-multus/0.log"
Jul 22 13:42:43.174293 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:43.174254    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mncs6_5b615de2-b59e-417c-95d5-42cb2c7fd256/network-metrics-daemon/0.log"
Jul 22 13:42:43.193815 ip-10-0-140-133 kubenswrapper[2575]: I0722 13:42:43.193721    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mncs6_5b615de2-b59e-417c-95d5-42cb2c7fd256/kube-rbac-proxy/0.log"