Jul 22 18:38:00.100279 ip-10-0-139-185 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 22 18:38:00.100289 ip-10-0-139-185 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 22 18:38:00.100296 ip-10-0-139-185 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 22 18:38:00.100535 ip-10-0-139-185 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 22 18:38:10.120029 ip-10-0-139-185 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 22 18:38:10.120047 ip-10-0-139-185 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e1a5af2d1f9844718ac18a2b21fcd59e --
Jul 22 18:39:59.002894 ip-10-0-139-185 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.367865    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371930    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371942    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371946    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371950    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371953    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371956    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371959    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:39:59.379616 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371961    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371964    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371967    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371970    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371973    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371976    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371978    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371981    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371984    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371986    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371989    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371992    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371995    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.371998    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372000    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372003    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372006    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372009    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372012    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:39:59.394527 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372022    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372027    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372030    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372033    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372035    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372038    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372041    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372044    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372047    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372050    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372053    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372055    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372058    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372061    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372064    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372067    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372069    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372072    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372076    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372079    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:39:59.396283 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372081    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372084    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372087    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372090    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372092    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372095    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372098    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372100    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372103    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372105    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372109    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372113    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372120    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372123    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372126    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372130    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372133    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372136    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372139    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:39:59.398636 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372142    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372145    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372148    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372151    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372155    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372157    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372160    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372162    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372165    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372167    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372170    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372173    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372176    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372178    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372181    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372184    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372186    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372189    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372191    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372194    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:39:59.410001 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372196    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372199    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372201    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372204    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372207    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372209    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372211    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372214    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372216    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372219    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372222    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372224    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372227    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372229    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372927    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372935    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372939    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372949    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372952    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372955    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:39:59.412309 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372958    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372961    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372963    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372966    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372969    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372972    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372976    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372980    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372984    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372987    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372990    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372992    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372995    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.372997    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373000    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373003    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373005    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373008    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373013    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:39:59.479539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373016    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:39:59.436189 ip-10-0-139-185 systemd[1]: Started Kubernetes Kubelet.
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373018    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373021    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373023    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373026    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373030    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373033    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373036    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373039    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373042    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373045    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373048    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373050    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373053    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373056    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373059    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373061    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373064    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:39:59.513401 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373066    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373069    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373071    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373074    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373077    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373079    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373082    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373085    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373088    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373090    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373093    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373095    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373099    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373102    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373105    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373108    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373110    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373113    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373115    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373118    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:39:59.682759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373120    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373123    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373125    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373128    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373131    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373133    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373135    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373138    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373141    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373144    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373146    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373149    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373152    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373154    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373157    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373159    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373162    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373167    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373170    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373172    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:39:59.684114 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373175    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373177    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373179    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373182    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373184    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373189    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373192    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373195    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373197    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373200    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373202    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373205    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373207    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373213    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373215    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373218    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373575    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373579    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373582    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373584    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:39:59.685550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373587    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373590    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373594    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373597    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373600    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373602    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373605    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373607    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373610    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373613    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373615    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373618    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373621    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373638    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373641    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373644    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373646    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373651    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373653    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373656    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:39:59.686736 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373659    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373661    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373664    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373666    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373669    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373673    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373675    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373678    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373681    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373683    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373686    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373689    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373691    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373694    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373696    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373699    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373702    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373704    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373707    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373709    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:39:59.687332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373712    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373714    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373717    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373720    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373722    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373727    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373730    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373734    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373736    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373741    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373744    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373747    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373749    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373752    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373755    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373757    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373760    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373763    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373766    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:39:59.687974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373769    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373771    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373774    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373776    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373779    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373781    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373783    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373786    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373788    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373791    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373794    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373796    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373799    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373801    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373804    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373809    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373813    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373816    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373819    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:39:59.688538 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373822    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373825    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373827    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373832    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373834    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373837    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373840    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373843    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373845    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373848    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373850    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373854    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373857    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373859    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373861    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373864    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.373866    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373935    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373942    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373956    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373960    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 18:39:59.689219 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373964    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373967    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373971    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373975    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373978    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373981    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373984    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373987    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373990    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373994    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.373997    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374000    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374002    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374005    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374011    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374014    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374017    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374020    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374023    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374027    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374030    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374033    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374037    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374040    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 18:39:59.689853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374043    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374046    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374049    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374052    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374056    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374059    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374062    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374065    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374068    2576 flags.go:64] FLAG: --enable-server="true"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374070    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374074    2576 flags.go:64] FLAG: --event-burst="100"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374077    2576 flags.go:64] FLAG: --event-qps="50"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374080    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374083    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374086    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374089    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374092    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374095    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374098    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374101    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374104    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374108    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374112    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374115    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374118    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 18:39:59.690725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374121    2576 flags.go:64] FLAG: --feature-gates=""
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374124    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374127    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374130    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374133    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374137    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374140    2576 flags.go:64] FLAG: --help="false"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374143    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374146    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374149    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374152    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374156    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374159    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374162    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374164    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374167    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374170    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374173    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374176    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374178    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374181    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374184    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374187    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374190    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 18:39:59.692377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374193    2576 flags.go:64] FLAG: --lock-file=""
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374195    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374199    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374202    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374207    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374211    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374214    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374217    2576 flags.go:64] FLAG: --logging-format="text"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374219    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374222    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374225    2576 flags.go:64] FLAG: --manifest-url=""
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374228    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374232    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374236    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374240    2576 flags.go:64] FLAG: --max-pods="110"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374243    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374246    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374249    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374252    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374255    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374258    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374261    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374268    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374270    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374273    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 18:39:59.693709 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374276    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374279    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374284    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374287    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374290    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374293    2576 flags.go:64] FLAG: --port="10250"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374296    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374299    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b016630912d6000c"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374302    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374304    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374308    2576 flags.go:64] FLAG: --register-node="true"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374310    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374314    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374318    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374321    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374323    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374327    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374330    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374333    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374336    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374340    2576 flags.go:64] FLAG: --runonce="false"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374343    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374346    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374349    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374352    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 18:39:59.695944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374354    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374357    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374360    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374363    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374366    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374369    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374372    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374374    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374377    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374380    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374383    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374388    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374390    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374393    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374398    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374401    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374403    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374406    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374409    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374412    2576 flags.go:64] FLAG: --v="2"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374417    2576 flags.go:64] FLAG: --version="false"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374421    2576 flags.go:64] FLAG: --vmodule=""
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374424    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.374428    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 18:39:59.698135 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374541    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374547    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374550    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374555    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374558    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374562    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374565    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374568    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374571    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374574    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374577    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374580    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374582    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374586    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374588    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374591    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374593    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374596    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374598    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:39:59.700318 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374601    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374604    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374606    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374608    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374611    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374614    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374616    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374619    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374622    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374639    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374642    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374644    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374647    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374649    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374652    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374655    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374659    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374661    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374664    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374667    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:39:59.701388 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374670    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374673    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374676    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374678    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374681    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374684    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374687    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374689    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374692    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374694    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374697    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374700    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374702    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374705    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374707    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374710    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374712    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374715    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374718    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374720    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:39:59.702997 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374723    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374727    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374731    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374733    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374736    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374739    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374741    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374744    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374749    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374752    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374754    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374757    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374759    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374762    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374765    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374767    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374770    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374772    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374775    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374778    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:39:59.704414 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374780    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374783    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374786    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374789    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374791    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374794    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374796    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374799    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374801    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374804    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374806    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374809    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374811    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374815    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374818    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374821    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374824    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374827    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374829    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:39:59.705535 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.374832    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.375550    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.381712    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.381725    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.383155    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.383163    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383218    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383223    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383226    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383230    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383232    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383235    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383238    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383241    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383243    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:39:59.706363 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383246    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383248    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383251    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383254    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383256    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383259    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383262    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383264    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383267    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383269    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383272    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383274    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383277    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383279    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383282    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383285    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383288    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383291    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383294    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383296    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:39:59.710184 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383300    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383302    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383305    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383307    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383310    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383312    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383315    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383318    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383320    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383323    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383326    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383328    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383331    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383333    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383336    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383338    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383341    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383343    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383346    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383348    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:39:59.711060 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383351    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383353    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383356    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383358    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383360    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383363    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383365    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383368    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383370    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383373    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383376    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383378    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383381    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383384    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383386    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383389    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383393    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383397    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383401    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:39:59.855337 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383404    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383408    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383411    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383414    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383417    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383419    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383422    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383425    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383428    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383431    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383434    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383438    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383441    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383444    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383447    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383449    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383452    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383454    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383457    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:39:59.856435 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383459    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383462    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383464    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383467    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383470    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383472    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383475    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383477    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383480    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383483    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383486    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383488    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.383493    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383596    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383601    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:39:59.857333 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383604    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383607    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383610    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383613    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383616    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383619    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383621    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383636    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383639    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383643    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383646    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383649    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383656    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383659    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383662    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383664    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383667    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383670    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383672    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383675    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:39:59.858017 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383677    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383680    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383683    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383686    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383689    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383692    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383695    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383697    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383700    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383702    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383705    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383708    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383710    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383713    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383716    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383720    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383723    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383725    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383728    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:39:59.858907 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383732    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383735    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383740    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383744    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383748    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383753    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383756    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383759    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383762    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383765    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383767    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383770    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383772    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383775    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383777    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383780    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383782    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383785    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383787    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:39:59.859798 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383790    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383793    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383795    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383798    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383800    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383803    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383805    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383807    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383810    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383812    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383815    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383817    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383820    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383822    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383824    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383829    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383831    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383834    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383838    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383841    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:39:59.860885 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383843    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383846    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383849    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383851    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383854    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383856    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383859    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383861    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383864    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383866    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383869    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383871    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383874    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383876    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383879    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383882    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383884    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383887    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:39:59.861780 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:39:59.383889    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.383893    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.383982    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.385757    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.387039    2576 server.go:1019] "Starting client certificate rotation"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.387136    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.387173    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.389051    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.394652    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.404722    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.406511    2576 log.go:25] "Validated CRI v1 image API"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.407178    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 18:39:59.862675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.412714    2576 fs.go:135] Filesystem UUIDs: map[78320bb2-acb3-41a5-a69c-abca074e6064:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 af68d250-2ae9-4a95-849b-e57d6856cae2:/dev/nvme0n1p3]
Jul 22 18:39:59.863325 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.412737    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 18:39:59.863325 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.414409    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.417667    2576 manager.go:216] Machine: {Timestamp:2026-07-22 18:39:59.41638112 +0000 UTC m=+0.306922058 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101437 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2527a83ad714da36446a09da87efdc SystemUUID:ec2527a8-3ad7-14da-3644-6a09da87efdc BootID:e1a5af2d-1f98-4471-8ac1-8a2b21fcd59e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3f:dd:18:a2:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3f:dd:18:a2:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:86:ef:24:b0:b8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.417764    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.417837    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.418673    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.418693    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-185.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.419340    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.419349    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.419356    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.419371    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.420380    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.423304    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.423403    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.423428    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.423437    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.423450    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.423458    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.423473    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 18:39:59.863986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.423772    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.423786    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.426354    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.430723    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.430754    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431613    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431644    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431653    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431661    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431674    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431682    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431692    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431697    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431705    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431710    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431719    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.431724    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.432151    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.432160    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.434372    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gz9wk"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.435397    2576 server.go:1268] "Started kubelet"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.435415    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.435483    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.435516    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.435900    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-185.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.436008    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.436056    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.437080    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.437315    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.441771    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.441835    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.441852    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.442286    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.442485    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.442499    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.442734    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.442742    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.442220    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-185.ec2.internal.18c4b0aaacc6d734  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-185.ec2.internal,UID:ip-10-0-139-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-185.ec2.internal,},FirstTimestamp:2026-07-22 18:39:59.435200308 +0000 UTC m=+0.325741230,LastTimestamp:2026-07-22 18:39:59.435200308 +0000 UTC m=+0.325741230,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-185.ec2.internal,}"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.443742    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.443930    2576 factory.go:55] Registering systemd factory
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.443963    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 22 18:39:59.864869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.444387    2576 factory.go:153] Registering CRI-O factory
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.444400    2576 factory.go:223] Registration of the crio container factory successfully
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.444452    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.444478    2576 factory.go:103] Registering Raw factory
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.444491    2576 manager.go:1196] Started watching for new ooms in manager
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.444863    2576 manager.go:349] Starting recovery of all containers
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.445391    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gz9wk"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.448393    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.448489    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.455662    2576 manager.go:354] Recovery completed
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.459705    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.461486    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.461511    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.461523    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.462926    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.462933    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.462936    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.462986    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.463913    2576 policy_none.go:49] "None policy: Start"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.463927    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.463941    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.464009    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.464029    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.464044    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.465037    2576 policy_none.go:47] "Start"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.535330    2576 manager.go:337] "Starting Device Plugin manager"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.535369    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.535382    2576 server.go:87] "Starting device plugin registration server"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.535714    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.535727    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.535846    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.535962    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.535971    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.536450    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.536501    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.560112    2576 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2610 into the system container \"/system.slice\": <nil>"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.636733    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.637581    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.637611    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.637623    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.637663    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.645029    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.645047    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-185.ec2.internal\": node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.656493    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:39:59.866919 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.680267    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.681505    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.681552    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.681577    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.681615    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.683237    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.756754    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.781896    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal"]
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.781987    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.782872    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.782900    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.782913    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.784168    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.784277    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.784301    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.791510    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.791540    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.791549    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.791515    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.791618    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.791647    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.792654    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.792676    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.793402    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.793424    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.793436    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.834727    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-185.ec2.internal\" not found" node="ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.839357    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-185.ec2.internal\" not found" node="ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.845596    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/96dfa4855b2f1801c68e37c41554c30b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal\" (UID: \"96dfa4855b2f1801c68e37c41554c30b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.845616    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96dfa4855b2f1801c68e37c41554c30b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal\" (UID: \"96dfa4855b2f1801c68e37c41554c30b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.845648    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/56d73fc2a3618a7351db3712287acb62-config\") pod \"kube-apiserver-proxy-ip-10-0-139-185.ec2.internal\" (UID: \"56d73fc2a3618a7351db3712287acb62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal"
Jul 22 18:39:59.868924 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.857654    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.049286 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.946328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/56d73fc2a3618a7351db3712287acb62-config\") pod \"kube-apiserver-proxy-ip-10-0-139-185.ec2.internal\" (UID: \"56d73fc2a3618a7351db3712287acb62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.049286 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.946363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/96dfa4855b2f1801c68e37c41554c30b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal\" (UID: \"96dfa4855b2f1801c68e37c41554c30b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.049286 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.946379    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96dfa4855b2f1801c68e37c41554c30b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal\" (UID: \"96dfa4855b2f1801c68e37c41554c30b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.049286 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.946433    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/56d73fc2a3618a7351db3712287acb62-config\") pod \"kube-apiserver-proxy-ip-10-0-139-185.ec2.internal\" (UID: \"56d73fc2a3618a7351db3712287acb62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.049286 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.946502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96dfa4855b2f1801c68e37c41554c30b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal\" (UID: \"96dfa4855b2f1801c68e37c41554c30b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.049286 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:39:59.946536    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/96dfa4855b2f1801c68e37c41554c30b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal\" (UID: \"96dfa4855b2f1801c68e37c41554c30b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.049286 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:39:59.958417    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.184164 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:00.058892    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.184164 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.138086    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.184164 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.141776    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.184164 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:00.159394    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.351042 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:00.260282    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.484764 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:00.360448    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.484764 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.388701    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 18:40:00.484764 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.388843    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 18:40:00.484764 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.448808    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 18:35:00 +0000 UTC" deadline="2028-03-20 23:10:39.068308766 +0000 UTC"
Jul 22 18:40:00.484764 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.448829    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14572h30m38.619482395s"
Jul 22 18:40:00.484764 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:00.461549    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.638100 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:00.562586    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.638100 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.596730    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:40:00.814513 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:00.663619    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.814513 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:00.764312    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-185.ec2.internal\" not found"
Jul 22 18:40:00.822363 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.821683    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:40:00.969550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.829065    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:40:00.969550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.842261    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.969550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.855454    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 18:40:00.969550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.856418    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal"
Jul 22 18:40:00.969550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.863399    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 18:40:00.969550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:40:00.909944    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d73fc2a3618a7351db3712287acb62.slice/crio-e17d78e8fc05cce5f6b1d035d9629dc24abb548c96739d761a87e747d6cf21df WatchSource:0}: Error finding container e17d78e8fc05cce5f6b1d035d9629dc24abb548c96739d761a87e747d6cf21df: Status 404 returned error can't find the container with id e17d78e8fc05cce5f6b1d035d9629dc24abb548c96739d761a87e747d6cf21df
Jul 22 18:40:00.969550 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:40:00.910509    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96dfa4855b2f1801c68e37c41554c30b.slice/crio-217566c056d816211b7ab29bdcd728442b37aefa2abd3f35b10a3ade63d5c629 WatchSource:0}: Error finding container 217566c056d816211b7ab29bdcd728442b37aefa2abd3f35b10a3ade63d5c629: Status 404 returned error can't find the container with id 217566c056d816211b7ab29bdcd728442b37aefa2abd3f35b10a3ade63d5c629
Jul 22 18:40:00.969550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:00.917220    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.424259    2576 apiserver.go:52] "Watching apiserver"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.434392    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.435487    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-xd9sx","openshift-network-diagnostics/network-check-target-6wvxw","openshift-network-operator/iptables-alerter-88b78","openshift-ovn-kubernetes/ovnkube-node-c88z2","kube-system/global-pull-secret-syncer-shh65","kube-system/konnectivity-agent-nxss6","kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal","openshift-cluster-node-tuning-operator/tuned-l5mbl","openshift-dns/node-resolver-4kzc7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal","openshift-multus/multus-additional-cni-plugins-qzlbl","openshift-multus/multus-wgr2l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn","openshift-image-registry/node-ca-4p574"]
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.437140    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.438758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.438824    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.440045    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.441380    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.441417    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.441551    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.441768    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.441778    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cvqqv\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.442360    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.442801    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.442870    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.443023    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5dxfn\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.443066    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.443325    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.443379    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.448636    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.448745    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.448935    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qjgq6\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.449056    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.449165    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.449389    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.449394    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.449962    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.452556    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.452678    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:01.476550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.452834    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.453000    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sk2ts\""
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.453104    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.454726    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be0cb195-07a4-41f1-abb8-80b1bc1a2046-host-slash\") pod \"iptables-alerter-88b78\" (UID: \"be0cb195-07a4-41f1-abb8-80b1bc1a2046\") " pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-run-ovn\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd157c08-d963-4c18-a43d-16418f13baf4-env-overrides\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-run-openvswitch\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455408    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd157c08-d963-4c18-a43d-16418f13baf4-ovnkube-script-lib\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jxcw\" (UniqueName: \"kubernetes.io/projected/be0cb195-07a4-41f1-abb8-80b1bc1a2046-kube-api-access-2jxcw\") pod \"iptables-alerter-88b78\" (UID: \"be0cb195-07a4-41f1-abb8-80b1bc1a2046\") " pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-run-systemd\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-log-socket\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-run-ovn-kubernetes\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455563    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-cni-netd\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455597    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-etc-selinux\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455619    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-sys-fs\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455640    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-run-netns\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.479582 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455699    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-dbus\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bcmz\" (UniqueName: \"kubernetes.io/projected/bd157c08-d963-4c18-a43d-16418f13baf4-kube-api-access-7bcmz\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jbd5\" (UniqueName: \"kubernetes.io/projected/df97a58c-fc62-4399-aa26-a117c7bd5f5c-kube-api-access-7jbd5\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-kubelet\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-systemd-units\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-node-log\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-cni-bin\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd157c08-d963-4c18-a43d-16418f13baf4-ovnkube-config\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.455982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456012    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-registration-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-kubelet-config\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456110    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-slash\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456150    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd157c08-d963-4c18-a43d-16418f13baf4-ovn-node-metrics-cert\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.480691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456243    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-socket-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456275    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-device-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-var-lib-openvswitch\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-etc-openvswitch\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456396    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be0cb195-07a4-41f1-abb8-80b1bc1a2046-iptables-alerter-script\") pod \"iptables-alerter-88b78\" (UID: \"be0cb195-07a4-41f1-abb8-80b1bc1a2046\") " pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.456867    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.457298    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.457333    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.457556    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6pdjq\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.457804    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.458830    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4sqbs\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.459007    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.459035    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.459253    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.460075    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.460174    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.460482    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.460520    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.460656    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9qjfp\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.461695    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.462102    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.462203    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.462688    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q86fk\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.462884    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.463280    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.463951    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.464196    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rcmcf\""
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.479905    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:40:01.482002 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.480064    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b49pq"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.486182    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b49pq"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.544065    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.556797    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-sysctl-d\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.556834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-sysctl-conf\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.556863    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-systemd\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.556888    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-sys\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.556933    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/89019e31-6d29-49fd-8fc3-f486fd2c1209-serviceca\") pod \"node-ca-4p574\" (UID: \"89019e31-6d29-49fd-8fc3-f486fd2c1209\") " pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.556991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-device-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-lib-modules\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-device-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-os-release\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-run-ovn\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd157c08-d963-4c18-a43d-16418f13baf4-env-overrides\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-cnibin\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-os-release\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557245    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-run-ovn\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab-hosts-file\") pod \"node-resolver-4kzc7\" (UID: \"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab\") " pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.561542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557300    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-run-openvswitch\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-run\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-run-openvswitch\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557396    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-var-lib-cni-multus\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557439    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-run-multus-certs\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f88707f9-0289-4032-a079-321f82e7c4e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-log-socket\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-run-ovn-kubernetes\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-run-ovn-kubernetes\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-cni-netd\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557612    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-log-socket\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-etc-selinux\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557686    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-cni-netd\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/89019e31-6d29-49fd-8fc3-f486fd2c1209-host\") pod \"node-ca-4p574\" (UID: \"89019e31-6d29-49fd-8fc3-f486fd2c1209\") " pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd157c08-d963-4c18-a43d-16418f13baf4-env-overrides\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557764    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-cni-dir\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-run-k8s-cni-cncf-io\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.566604 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557830    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-etc-selinux\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557917    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-dbus\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f88707f9-0289-4032-a079-321f82e7c4e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.557977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a6d9e14-95eb-4686-ab67-5ca7aff3011e-konnectivity-ca\") pod \"konnectivity-agent-nxss6\" (UID: \"3a6d9e14-95eb-4686-ab67-5ca7aff3011e\") " pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-kubernetes\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-host\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558167    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-tuned\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnckb\" (UniqueName: \"kubernetes.io/projected/ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab-kube-api-access-bnckb\") pod \"node-resolver-4kzc7\" (UID: \"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab\") " pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-dbus\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558247    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/412b6a94-062d-4f4f-a8b7-0c031cb16908-cni-binary-copy\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-hostroot\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558306    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-kubelet\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558331    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-node-log\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-kubelet\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd157c08-d963-4c18-a43d-16418f13baf4-ovnkube-config\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.567993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558380    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-node-log\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558410    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-registration-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558427    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab-tmp-dir\") pod \"node-resolver-4kzc7\" (UID: \"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab\") " pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-run-netns\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.558463    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-etc-kubernetes\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558508    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-registration-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.558564    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret podName:04e18ad9-ab77-4ea3-b3b7-3c2a570e096b nodeName:}" failed. No retries permitted until 2026-07-22 18:40:02.058541056 +0000 UTC m=+2.949081974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret") pod "global-pull-secret-syncer-shh65" (UID: "04e18ad9-ab77-4ea3-b3b7-3c2a570e096b") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558619    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-kubelet-config\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558717    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggwxn\" (UniqueName: \"kubernetes.io/projected/f88707f9-0289-4032-a079-321f82e7c4e0-kube-api-access-ggwxn\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-kubelet-config\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd157c08-d963-4c18-a43d-16418f13baf4-ovn-node-metrics-cert\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.569417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-socket-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a6d9e14-95eb-4686-ab67-5ca7aff3011e-agent-certs\") pod \"konnectivity-agent-nxss6\" (UID: \"3a6d9e14-95eb-4686-ab67-5ca7aff3011e\") " pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558886    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb2cq\" (UniqueName: \"kubernetes.io/projected/89019e31-6d29-49fd-8fc3-f486fd2c1209-kube-api-access-pb2cq\") pod \"node-ca-4p574\" (UID: \"89019e31-6d29-49fd-8fc3-f486fd2c1209\") " pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd157c08-d963-4c18-a43d-16418f13baf4-ovnkube-config\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-daemon-config\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558938    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-var-lib-openvswitch\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558964    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-etc-openvswitch\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be0cb195-07a4-41f1-abb8-80b1bc1a2046-iptables-alerter-script\") pod \"iptables-alerter-88b78\" (UID: \"be0cb195-07a4-41f1-abb8-80b1bc1a2046\") " pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.558993    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-socket-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559015    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-var-lib-openvswitch\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559024    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be0cb195-07a4-41f1-abb8-80b1bc1a2046-host-slash\") pod \"iptables-alerter-88b78\" (UID: \"be0cb195-07a4-41f1-abb8-80b1bc1a2046\") " pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559050    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-etc-openvswitch\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559062    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-sysconfig\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559071    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be0cb195-07a4-41f1-abb8-80b1bc1a2046-host-slash\") pod \"iptables-alerter-88b78\" (UID: \"be0cb195-07a4-41f1-abb8-80b1bc1a2046\") " pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559076    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-var-lib-kubelet\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559119    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz85r\" (UniqueName: \"kubernetes.io/projected/412b6a94-062d-4f4f-a8b7-0c031cb16908-kube-api-access-dz85r\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.573475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd157c08-d963-4c18-a43d-16418f13baf4-ovnkube-script-lib\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5cq5\" (UniqueName: \"kubernetes.io/projected/881e051a-b4aa-4d47-b389-0f3dd300de2f-kube-api-access-v5cq5\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559260    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8qtz\" (UniqueName: \"kubernetes.io/projected/02cd0ca5-45f3-436b-ae62-381b2a755145-kube-api-access-n8qtz\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-system-cni-dir\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxcw\" (UniqueName: \"kubernetes.io/projected/be0cb195-07a4-41f1-abb8-80b1bc1a2046-kube-api-access-2jxcw\") pod \"iptables-alerter-88b78\" (UID: \"be0cb195-07a4-41f1-abb8-80b1bc1a2046\") " pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559351    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-run-systemd\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-sys-fs\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-system-cni-dir\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-var-lib-kubelet\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559466    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-cnibin\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559482    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-sys-fs\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559490    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-run-netns\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-run-systemd\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-modprobe-d\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-conf-dir\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bcmz\" (UniqueName: \"kubernetes.io/projected/bd157c08-d963-4c18-a43d-16418f13baf4-kube-api-access-7bcmz\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559570    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-run-netns\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.574634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jbd5\" (UniqueName: \"kubernetes.io/projected/df97a58c-fc62-4399-aa26-a117c7bd5f5c-kube-api-access-7jbd5\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-socket-dir-parent\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559639    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-systemd-units\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-cni-bin\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-systemd-units\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f88707f9-0289-4032-a079-321f82e7c4e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd157c08-d963-4c18-a43d-16418f13baf4-ovnkube-script-lib\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559749    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02cd0ca5-45f3-436b-ae62-381b2a755145-tmp\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559773    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-var-lib-cni-bin\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-slash\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559831    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.559774    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-cni-bin\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.565149    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd157c08-d963-4c18-a43d-16418f13baf4-ovn-node-metrics-cert\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.565219    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.565236    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.565247    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kwj4v for pod openshift-network-diagnostics/network-check-target-6wvxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.573027    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jbd5\" (UniqueName: \"kubernetes.io/projected/df97a58c-fc62-4399-aa26-a117c7bd5f5c-kube-api-access-7jbd5\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.574147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jxcw\" (UniqueName: \"kubernetes.io/projected/be0cb195-07a4-41f1-abb8-80b1bc1a2046-kube-api-access-2jxcw\") pod \"iptables-alerter-88b78\" (UID: \"be0cb195-07a4-41f1-abb8-80b1bc1a2046\") " pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.576068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.575432    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bcmz\" (UniqueName: \"kubernetes.io/projected/bd157c08-d963-4c18-a43d-16418f13baf4-kube-api-access-7bcmz\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.718478 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.688047    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd157c08-d963-4c18-a43d-16418f13baf4-host-slash\") pod \"ovnkube-node-c88z2\" (UID: \"bd157c08-d963-4c18-a43d-16418f13baf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.718478 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.688049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/df97a58c-fc62-4399-aa26-a117c7bd5f5c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cwvtn\" (UID: \"df97a58c-fc62-4399-aa26-a117c7bd5f5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.718478 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.689240    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v podName:673d08df-5491-439e-b8de-0a21dd0b72f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:02.189184355 +0000 UTC m=+3.079725276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kwj4v" (UniqueName: "kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v") pod "network-check-target-6wvxw" (UID: "673d08df-5491-439e-b8de-0a21dd0b72f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:01.718478 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.689277    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be0cb195-07a4-41f1-abb8-80b1bc1a2046-iptables-alerter-script\") pod \"iptables-alerter-88b78\" (UID: \"be0cb195-07a4-41f1-abb8-80b1bc1a2046\") " pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.718478 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.692345    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal" event={"ID":"56d73fc2a3618a7351db3712287acb62","Type":"ContainerStarted","Data":"e17d78e8fc05cce5f6b1d035d9629dc24abb548c96739d761a87e747d6cf21df"}
Jul 22 18:40:01.718478 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.693742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal" event={"ID":"96dfa4855b2f1801c68e37c41554c30b","Type":"ContainerStarted","Data":"217566c056d816211b7ab29bdcd728442b37aefa2abd3f35b10a3ade63d5c629"}
Jul 22 18:40:01.758159 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.758133    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn"
Jul 22 18:40:01.770777 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.769401    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:01.783310 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.782839    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-88b78"
Jul 22 18:40:01.788138 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-os-release\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.788226 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-cnibin\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.788226 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-os-release\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.788226 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab-hosts-file\") pod \"node-resolver-4kzc7\" (UID: \"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab\") " pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.788403 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-run\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.788403 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-var-lib-cni-multus\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.788403 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-run-multus-certs\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.788403 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f88707f9-0289-4032-a079-321f82e7c4e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.788403 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/89019e31-6d29-49fd-8fc3-f486fd2c1209-host\") pod \"node-ca-4p574\" (UID: \"89019e31-6d29-49fd-8fc3-f486fd2c1209\") " pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.788403 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-cni-dir\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.788403 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-run-k8s-cni-cncf-io\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788429    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f88707f9-0289-4032-a079-321f82e7c4e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a6d9e14-95eb-4686-ab67-5ca7aff3011e-konnectivity-ca\") pod \"konnectivity-agent-nxss6\" (UID: \"3a6d9e14-95eb-4686-ab67-5ca7aff3011e\") " pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-kubernetes\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-host\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-tuned\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnckb\" (UniqueName: \"kubernetes.io/projected/ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab-kube-api-access-bnckb\") pod \"node-resolver-4kzc7\" (UID: \"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab\") " pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/412b6a94-062d-4f4f-a8b7-0c031cb16908-cni-binary-copy\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-hostroot\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788673    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab-tmp-dir\") pod \"node-resolver-4kzc7\" (UID: \"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab\") " pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-run-netns\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-etc-kubernetes\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.788773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788806    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggwxn\" (UniqueName: \"kubernetes.io/projected/f88707f9-0289-4032-a079-321f82e7c4e0-kube-api-access-ggwxn\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a6d9e14-95eb-4686-ab67-5ca7aff3011e-agent-certs\") pod \"konnectivity-agent-nxss6\" (UID: \"3a6d9e14-95eb-4686-ab67-5ca7aff3011e\") " pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb2cq\" (UniqueName: \"kubernetes.io/projected/89019e31-6d29-49fd-8fc3-f486fd2c1209-kube-api-access-pb2cq\") pod \"node-ca-4p574\" (UID: \"89019e31-6d29-49fd-8fc3-f486fd2c1209\") " pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-daemon-config\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788917    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-sysconfig\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788942    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-var-lib-kubelet\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz85r\" (UniqueName: \"kubernetes.io/projected/412b6a94-062d-4f4f-a8b7-0c031cb16908-kube-api-access-dz85r\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.788991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5cq5\" (UniqueName: \"kubernetes.io/projected/881e051a-b4aa-4d47-b389-0f3dd300de2f-kube-api-access-v5cq5\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8qtz\" (UniqueName: \"kubernetes.io/projected/02cd0ca5-45f3-436b-ae62-381b2a755145-kube-api-access-n8qtz\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789025    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-kubernetes\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-system-cni-dir\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789073    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-system-cni-dir\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789097    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-var-lib-kubelet\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-cnibin\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-modprobe-d\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-conf-dir\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.789400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789199    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-os-release\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-socket-dir-parent\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f88707f9-0289-4032-a079-321f82e7c4e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789244    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-cnibin\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789267    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02cd0ca5-45f3-436b-ae62-381b2a755145-tmp\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789295    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-var-lib-cni-bin\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789298    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-os-release\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789330    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-sysctl-d\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789347    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab-hosts-file\") pod \"node-resolver-4kzc7\" (UID: \"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab\") " pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-sysctl-conf\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-systemd\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-run\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789411    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-sys\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789436    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-var-lib-cni-multus\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/89019e31-6d29-49fd-8fc3-f486fd2c1209-serviceca\") pod \"node-ca-4p574\" (UID: \"89019e31-6d29-49fd-8fc3-f486fd2c1209\") " pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-run-multus-certs\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-lib-modules\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789650    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-lib-modules\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.790237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/89019e31-6d29-49fd-8fc3-f486fd2c1209-serviceca\") pod \"node-ca-4p574\" (UID: \"89019e31-6d29-49fd-8fc3-f486fd2c1209\") " pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.790987 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.789981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-host\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.790987 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.790246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f88707f9-0289-4032-a079-321f82e7c4e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.790987 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.790313    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/89019e31-6d29-49fd-8fc3-f486fd2c1209-host\") pod \"node-ca-4p574\" (UID: \"89019e31-6d29-49fd-8fc3-f486fd2c1209\") " pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.790987 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.790367    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-run-k8s-cni-cncf-io\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.791248 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f88707f9-0289-4032-a079-321f82e7c4e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.791248 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791217    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-hostroot\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791501    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-sysctl-d\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-run-netns\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791667    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f88707f9-0289-4032-a079-321f82e7c4e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.791757    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:01.791808    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:02.291791238 +0000 UTC m=+3.182332166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791817    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-socket-dir-parent\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-modprobe-d\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-sysconfig\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-systemd\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791913    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-var-lib-kubelet\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791758    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f88707f9-0289-4032-a079-321f82e7c4e0-system-cni-dir\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791986    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab-tmp-dir\") pod \"node-resolver-4kzc7\" (UID: \"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab\") " pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.791990    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-conf-dir\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792048    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-etc-kubernetes\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-cni-dir\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.792227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-sysctl-conf\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.793168 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792242    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a6d9e14-95eb-4686-ab67-5ca7aff3011e-konnectivity-ca\") pod \"konnectivity-agent-nxss6\" (UID: \"3a6d9e14-95eb-4686-ab67-5ca7aff3011e\") " pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:01.793168 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792257    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-sys\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.793168 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792307    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-cnibin\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.793168 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792313    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-system-cni-dir\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.793168 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02cd0ca5-45f3-436b-ae62-381b2a755145-var-lib-kubelet\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.793168 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792411    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/412b6a94-062d-4f4f-a8b7-0c031cb16908-host-var-lib-cni-bin\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.793168 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792583    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/412b6a94-062d-4f4f-a8b7-0c031cb16908-multus-daemon-config\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.793168 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.792894    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/412b6a94-062d-4f4f-a8b7-0c031cb16908-cni-binary-copy\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.796169 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.796143    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a6d9e14-95eb-4686-ab67-5ca7aff3011e-agent-certs\") pod \"konnectivity-agent-nxss6\" (UID: \"3a6d9e14-95eb-4686-ab67-5ca7aff3011e\") " pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:01.796261 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:40:01.796232    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe0cb195_07a4_41f1_abb8_80b1bc1a2046.slice/crio-c9efd63157dee2d559af2f179039bb5351c6f6dd9ae2af7a9b58392cf0951380 WatchSource:0}: Error finding container c9efd63157dee2d559af2f179039bb5351c6f6dd9ae2af7a9b58392cf0951380: Status 404 returned error can't find the container with id c9efd63157dee2d559af2f179039bb5351c6f6dd9ae2af7a9b58392cf0951380
Jul 22 18:40:01.797894 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.797830    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02cd0ca5-45f3-436b-ae62-381b2a755145-tmp\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.798176 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.798136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02cd0ca5-45f3-436b-ae62-381b2a755145-etc-tuned\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.800408 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.799381    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnckb\" (UniqueName: \"kubernetes.io/projected/ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab-kube-api-access-bnckb\") pod \"node-resolver-4kzc7\" (UID: \"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab\") " pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.800635 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.800608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8qtz\" (UniqueName: \"kubernetes.io/projected/02cd0ca5-45f3-436b-ae62-381b2a755145-kube-api-access-n8qtz\") pod \"tuned-l5mbl\" (UID: \"02cd0ca5-45f3-436b-ae62-381b2a755145\") " pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:01.803320 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.803275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggwxn\" (UniqueName: \"kubernetes.io/projected/f88707f9-0289-4032-a079-321f82e7c4e0-kube-api-access-ggwxn\") pod \"multus-additional-cni-plugins-qzlbl\" (UID: \"f88707f9-0289-4032-a079-321f82e7c4e0\") " pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.803912 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.803890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb2cq\" (UniqueName: \"kubernetes.io/projected/89019e31-6d29-49fd-8fc3-f486fd2c1209-kube-api-access-pb2cq\") pod \"node-ca-4p574\" (UID: \"89019e31-6d29-49fd-8fc3-f486fd2c1209\") " pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.805811 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.805790    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz85r\" (UniqueName: \"kubernetes.io/projected/412b6a94-062d-4f4f-a8b7-0c031cb16908-kube-api-access-dz85r\") pod \"multus-wgr2l\" (UID: \"412b6a94-062d-4f4f-a8b7-0c031cb16908\") " pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.807477 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.807433    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5cq5\" (UniqueName: \"kubernetes.io/projected/881e051a-b4aa-4d47-b389-0f3dd300de2f-kube-api-access-v5cq5\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:01.807559 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.807506    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4kzc7"
Jul 22 18:40:01.817199 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.816777    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4p574"
Jul 22 18:40:01.818643 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:40:01.818620    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba8ebbb4_4eac_46c0_a872_5b3ea8d447ab.slice/crio-dcf1e5baf136688e3d85290c026f21d8fca59f8c9d2d73a27fd355f094b90b3f WatchSource:0}: Error finding container dcf1e5baf136688e3d85290c026f21d8fca59f8c9d2d73a27fd355f094b90b3f: Status 404 returned error can't find the container with id dcf1e5baf136688e3d85290c026f21d8fca59f8c9d2d73a27fd355f094b90b3f
Jul 22 18:40:01.823858 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.823675    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qzlbl"
Jul 22 18:40:01.831984 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:01.831963    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wgr2l"
Jul 22 18:40:01.836651 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:40:01.836607    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88707f9_0289_4032_a079_321f82e7c4e0.slice/crio-b5e450c355e98b2a4d93753fb2a2fd8a10720b55bda540ea4de7680f0741d15f WatchSource:0}: Error finding container b5e450c355e98b2a4d93753fb2a2fd8a10720b55bda540ea4de7680f0741d15f: Status 404 returned error can't find the container with id b5e450c355e98b2a4d93753fb2a2fd8a10720b55bda540ea4de7680f0741d15f
Jul 22 18:40:01.847332 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:40:01.847302    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod412b6a94_062d_4f4f_a8b7_0c031cb16908.slice/crio-6ece22ed505b8a4f5cf64e9c9c1aa87751834147f2c6b7a2f162b50a0fb07ab7 WatchSource:0}: Error finding container 6ece22ed505b8a4f5cf64e9c9c1aa87751834147f2c6b7a2f162b50a0fb07ab7: Status 404 returned error can't find the container with id 6ece22ed505b8a4f5cf64e9c9c1aa87751834147f2c6b7a2f162b50a0fb07ab7
Jul 22 18:40:02.089494 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.089404    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:02.091417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.091388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:02.091586 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:02.091564    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:02.091780 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:02.091691    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret podName:04e18ad9-ab77-4ea3-b3b7-3c2a570e096b nodeName:}" failed. No retries permitted until 2026-07-22 18:40:03.091621822 +0000 UTC m=+3.982162753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret") pod "global-pull-secret-syncer-shh65" (UID: "04e18ad9-ab77-4ea3-b3b7-3c2a570e096b") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:02.100948 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.100914    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l5mbl"
Jul 22 18:40:02.192212 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.192177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:02.192377 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:02.192318    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:40:02.192377 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:02.192336    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:40:02.192377 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:02.192346    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kwj4v for pod openshift-network-diagnostics/network-check-target-6wvxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:02.192523 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:02.192410    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v podName:673d08df-5491-439e-b8de-0a21dd0b72f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:03.19239412 +0000 UTC m=+4.082935032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwj4v" (UniqueName: "kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v") pod "network-check-target-6wvxw" (UID: "673d08df-5491-439e-b8de-0a21dd0b72f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:02.293325 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.293283    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:02.293481 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:02.293439    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:02.293547 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:02.293497    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:03.293483154 +0000 UTC m=+4.184024065 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:02.488356 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.488291    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:35:02 +0000 UTC" deadline="2028-02-18 14:37:49.396287169 +0000 UTC"
Jul 22 18:40:02.488356 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.488350    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13819h57m46.90794128s"
Jul 22 18:40:02.697345 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.697302    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qzlbl" event={"ID":"f88707f9-0289-4032-a079-321f82e7c4e0","Type":"ContainerStarted","Data":"b5e450c355e98b2a4d93753fb2a2fd8a10720b55bda540ea4de7680f0741d15f"}
Jul 22 18:40:02.698499 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.698472    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4kzc7" event={"ID":"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab","Type":"ContainerStarted","Data":"dcf1e5baf136688e3d85290c026f21d8fca59f8c9d2d73a27fd355f094b90b3f"}
Jul 22 18:40:02.699737 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.699714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" event={"ID":"bd157c08-d963-4c18-a43d-16418f13baf4","Type":"ContainerStarted","Data":"d16a8fcb567bb95cbdf75cc4118c56ddeea9cb4f28ee27731b4efc8002353eaf"}
Jul 22 18:40:02.700988 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.700960    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn" event={"ID":"df97a58c-fc62-4399-aa26-a117c7bd5f5c","Type":"ContainerStarted","Data":"ce11c63a9c89122c77552104f3fd65cb105ae7cad17c627b91ac0e28afa764c7"}
Jul 22 18:40:02.702159 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.702135    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wgr2l" event={"ID":"412b6a94-062d-4f4f-a8b7-0c031cb16908","Type":"ContainerStarted","Data":"6ece22ed505b8a4f5cf64e9c9c1aa87751834147f2c6b7a2f162b50a0fb07ab7"}
Jul 22 18:40:02.703357 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.703334    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4p574" event={"ID":"89019e31-6d29-49fd-8fc3-f486fd2c1209","Type":"ContainerStarted","Data":"54deff23153e2a435487f93a54638d0c9d476853f4985ec1ec9a898f762baf55"}
Jul 22 18:40:02.704540 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:02.704515    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-88b78" event={"ID":"be0cb195-07a4-41f1-abb8-80b1bc1a2046","Type":"ContainerStarted","Data":"c9efd63157dee2d559af2f179039bb5351c6f6dd9ae2af7a9b58392cf0951380"}
Jul 22 18:40:03.085566 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:40:03.085533    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02cd0ca5_45f3_436b_ae62_381b2a755145.slice/crio-55905ab250eb34f6e4da6fb685840d716bba049e950cc62c536c5611d05b9834 WatchSource:0}: Error finding container 55905ab250eb34f6e4da6fb685840d716bba049e950cc62c536c5611d05b9834: Status 404 returned error can't find the container with id 55905ab250eb34f6e4da6fb685840d716bba049e950cc62c536c5611d05b9834
Jul 22 18:40:03.100525 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.100470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:03.100604 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.100585    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:03.100709 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.100668    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret podName:04e18ad9-ab77-4ea3-b3b7-3c2a570e096b nodeName:}" failed. No retries permitted until 2026-07-22 18:40:05.100629996 +0000 UTC m=+5.991170908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret") pod "global-pull-secret-syncer-shh65" (UID: "04e18ad9-ab77-4ea3-b3b7-3c2a570e096b") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:03.201910 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.201444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:03.201910 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.201591    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:40:03.201910 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.201787    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:40:03.201910 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.201805    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kwj4v for pod openshift-network-diagnostics/network-check-target-6wvxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:03.201910 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.201862    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v podName:673d08df-5491-439e-b8de-0a21dd0b72f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:05.201843875 +0000 UTC m=+6.092384790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwj4v" (UniqueName: "kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v") pod "network-check-target-6wvxw" (UID: "673d08df-5491-439e-b8de-0a21dd0b72f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:03.302442 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.302365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:03.302654 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.302529    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:03.302654 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.302595    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:05.302577283 +0000 UTC m=+6.193118198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:03.488859 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.488791    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:35:02 +0000 UTC" deadline="2028-03-01 11:22:13.788351426 +0000 UTC"
Jul 22 18:40:03.488859 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.488850    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14104h42m10.299506333s"
Jul 22 18:40:03.685089 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.685048    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:03.685263 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.685193    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:03.685634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.685611    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:03.685760 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.685738    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:03.685824 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.685814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:03.685918 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:03.685900    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:03.723230 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.722969    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal" event={"ID":"56d73fc2a3618a7351db3712287acb62","Type":"ContainerStarted","Data":"7c373cfc9a0c89ab51d108960b15a287128f29d9dba1941c5f2b59cf7d0119f4"}
Jul 22 18:40:03.730778 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.730747    2576 generic.go:356] "Generic (PLEG): container finished" podID="96dfa4855b2f1801c68e37c41554c30b" containerID="1e0ba27b8c7c69b5087da2f4e7757bc5055282e7f16f83d924372ed346da4fe5" exitCode=0
Jul 22 18:40:03.730917 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.730839    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal" event={"ID":"96dfa4855b2f1801c68e37c41554c30b","Type":"ContainerDied","Data":"1e0ba27b8c7c69b5087da2f4e7757bc5055282e7f16f83d924372ed346da4fe5"}
Jul 22 18:40:03.737949 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.737922    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l5mbl" event={"ID":"02cd0ca5-45f3-436b-ae62-381b2a755145","Type":"ContainerStarted","Data":"55905ab250eb34f6e4da6fb685840d716bba049e950cc62c536c5611d05b9834"}
Jul 22 18:40:03.738368 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.738304    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-185.ec2.internal" podStartSLOduration=2.738285838 podStartE2EDuration="2.738285838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:03.737034655 +0000 UTC m=+4.627575592" watchObservedRunningTime="2026-07-22 18:40:03.738285838 +0000 UTC m=+4.628826776"
Jul 22 18:40:03.749178 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:03.749102    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nxss6" event={"ID":"3a6d9e14-95eb-4686-ab67-5ca7aff3011e","Type":"ContainerStarted","Data":"05c9107fd497c339d29157eeacd1ef43257eb66078dd112ee17a6d4044a96cad"}
Jul 22 18:40:04.765901 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:04.765804    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal" event={"ID":"96dfa4855b2f1801c68e37c41554c30b","Type":"ContainerStarted","Data":"9d5310b34c32766804ccf35324665468740f2eb4ca02a84457d7bb3aa02db02c"}
Jul 22 18:40:04.781275 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:04.781222    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-185.ec2.internal" podStartSLOduration=3.781201188 podStartE2EDuration="3.781201188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:04.780688457 +0000 UTC m=+5.671229393" watchObservedRunningTime="2026-07-22 18:40:04.781201188 +0000 UTC m=+5.671742124"
Jul 22 18:40:05.119934 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:05.119897    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:05.120106 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.120075    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:05.120183 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.120141    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret podName:04e18ad9-ab77-4ea3-b3b7-3c2a570e096b nodeName:}" failed. No retries permitted until 2026-07-22 18:40:09.120121631 +0000 UTC m=+10.010662548 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret") pod "global-pull-secret-syncer-shh65" (UID: "04e18ad9-ab77-4ea3-b3b7-3c2a570e096b") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:05.220958 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:05.220922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:05.221145 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.221108    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:40:05.221145 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.221130    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:40:05.221145 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.221144    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kwj4v for pod openshift-network-diagnostics/network-check-target-6wvxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:05.221304 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.221200    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v podName:673d08df-5491-439e-b8de-0a21dd0b72f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:09.221181575 +0000 UTC m=+10.111722494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwj4v" (UniqueName: "kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v") pod "network-check-target-6wvxw" (UID: "673d08df-5491-439e-b8de-0a21dd0b72f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:05.321623 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:05.321586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:05.321832 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.321761    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:05.321903 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.321835    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:09.321804034 +0000 UTC m=+10.212344960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:05.683841 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:05.683807    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:05.684030 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.683945    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:05.684278 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:05.684255    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:05.684382 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.684363    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:05.684436 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:05.684420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:05.684607 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:05.684581    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:07.685022 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:07.684965    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:07.685552 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:07.685102    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:07.685621 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:07.685556    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:07.685702 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:07.685660    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:07.685771 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:07.685747    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:07.685883 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:07.685862    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:09.157546 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:09.157504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:09.158044 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.157803    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:09.158044 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.157900    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret podName:04e18ad9-ab77-4ea3-b3b7-3c2a570e096b nodeName:}" failed. No retries permitted until 2026-07-22 18:40:17.157858291 +0000 UTC m=+18.048399212 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret") pod "global-pull-secret-syncer-shh65" (UID: "04e18ad9-ab77-4ea3-b3b7-3c2a570e096b") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:09.259187 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:09.258463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:09.259187 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.258730    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:40:09.259187 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.258752    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:40:09.259187 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.258766    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kwj4v for pod openshift-network-diagnostics/network-check-target-6wvxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:09.259187 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.258825    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v podName:673d08df-5491-439e-b8de-0a21dd0b72f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:17.258807084 +0000 UTC m=+18.149348001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwj4v" (UniqueName: "kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v") pod "network-check-target-6wvxw" (UID: "673d08df-5491-439e-b8de-0a21dd0b72f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:09.359390 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:09.359248    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:09.359563 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.359467    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:09.359563 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.359550    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:17.359526542 +0000 UTC m=+18.250067460 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:09.683644 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:09.683594    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:09.683839 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.683746    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:09.684084 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:09.684062    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:09.684216 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.684175    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:09.684216 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:09.684216    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:09.684391 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:09.684308    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:11.682696 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:11.682583    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:11.682696 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:11.682632    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:11.683275 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:11.682822    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:11.683275 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:11.682583    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:11.683395 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:11.683302    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:11.683463 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:11.683398    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:13.682847 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:13.682814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:13.682847 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:13.682843    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:13.683312 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:13.682815    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:13.683312 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:13.682940    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:13.683312 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:13.683032    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:13.683312 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:13.683134    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:15.682284 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:15.682246    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:15.682717 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:15.682246    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:15.682717 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:15.682401    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:15.682717 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:15.682413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:15.682717 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:15.682516    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:15.683521 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:15.682926    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:17.220287 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:17.220253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:17.220763 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.220413    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:17.220763 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.220496    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret podName:04e18ad9-ab77-4ea3-b3b7-3c2a570e096b nodeName:}" failed. No retries permitted until 2026-07-22 18:40:33.220473707 +0000 UTC m=+34.111014619 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret") pod "global-pull-secret-syncer-shh65" (UID: "04e18ad9-ab77-4ea3-b3b7-3c2a570e096b") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:17.321482 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:17.321452    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:17.321644 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.321603    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:40:17.321644 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.321621    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:40:17.321644 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.321645    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kwj4v for pod openshift-network-diagnostics/network-check-target-6wvxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:17.321793 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.321701    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v podName:673d08df-5491-439e-b8de-0a21dd0b72f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:33.321682698 +0000 UTC m=+34.212223627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwj4v" (UniqueName: "kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v") pod "network-check-target-6wvxw" (UID: "673d08df-5491-439e-b8de-0a21dd0b72f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:17.421814 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:17.421784    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:17.421991 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.421908    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:17.421991 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.421973    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:33.4219535 +0000 UTC m=+34.312494411 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:17.682565 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:17.682533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:17.682768 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:17.682533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:17.682768 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.682678    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:17.682768 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.682734    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:17.682768 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:17.682533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:17.682975 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:17.682821    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:19.683043 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:19.683009    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:19.683485 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:19.683122    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:19.683485 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:19.683225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:19.683485 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:19.683345    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:19.683485 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:19.683378    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:19.683485 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:19.683472    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:21.682239 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:21.682203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:21.682719 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:21.682204    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:21.682719 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:21.682337    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:21.682719 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:21.682424    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:21.682719 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:21.682217    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:21.682719 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:21.682540    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:22.807944 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.807664    2576 generic.go:356] "Generic (PLEG): container finished" podID="f88707f9-0289-4032-a079-321f82e7c4e0" containerID="57b1b1e7d3e8c3e4671fe402cb742e9ec2717a8aa43734dfec1e28d48ce9041c" exitCode=0
Jul 22 18:40:22.808598 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.807742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qzlbl" event={"ID":"f88707f9-0289-4032-a079-321f82e7c4e0","Type":"ContainerDied","Data":"57b1b1e7d3e8c3e4671fe402cb742e9ec2717a8aa43734dfec1e28d48ce9041c"}
Jul 22 18:40:22.809459 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.809435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4kzc7" event={"ID":"ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab","Type":"ContainerStarted","Data":"6aafce2bcfed6b828d51e07e1bfad66aae99498c80a415a41574938b3629ad92"}
Jul 22 18:40:22.811959 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.811937    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" event={"ID":"bd157c08-d963-4c18-a43d-16418f13baf4","Type":"ContainerStarted","Data":"81cf6bf9ab6410e2efa60c29cfba93a903e907e7d91f4520a9241b0a27254e51"}
Jul 22 18:40:22.812056 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.811994    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" event={"ID":"bd157c08-d963-4c18-a43d-16418f13baf4","Type":"ContainerStarted","Data":"085fc92ed7d399d3a942a31606af56242d66851848419befe1021eb66c7da232"}
Jul 22 18:40:22.812056 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.812038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" event={"ID":"bd157c08-d963-4c18-a43d-16418f13baf4","Type":"ContainerStarted","Data":"244be8328cf0e9432df7d66d68eaa04847d80ba2de867bb83b8497173250ea09"}
Jul 22 18:40:22.812164 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.812078    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" event={"ID":"bd157c08-d963-4c18-a43d-16418f13baf4","Type":"ContainerStarted","Data":"45ebb4936480c1303a1c30707cb769b2b47df115050f45659be14b2e939401ea"}
Jul 22 18:40:22.812164 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.812108    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" event={"ID":"bd157c08-d963-4c18-a43d-16418f13baf4","Type":"ContainerStarted","Data":"e9b7e7cea0de949c64ca3c097c60f11d3be2bee97aba02968bd7c0d087e15562"}
Jul 22 18:40:22.813315 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.813296    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn" event={"ID":"df97a58c-fc62-4399-aa26-a117c7bd5f5c","Type":"ContainerStarted","Data":"f2e3b8571512ff8da689fc75c650673d68dbbe66322750b75d9fbcb4af1068d5"}
Jul 22 18:40:22.814745 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.814721    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nxss6" event={"ID":"3a6d9e14-95eb-4686-ab67-5ca7aff3011e","Type":"ContainerStarted","Data":"bf717e2b061d446e7ca0deb762341f12d00fe229b6c153fdab8fc1ef93f6fe09"}
Jul 22 18:40:22.816197 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.816164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wgr2l" event={"ID":"412b6a94-062d-4f4f-a8b7-0c031cb16908","Type":"ContainerStarted","Data":"24485e071f26ea0029a6232d62f87e68d3bbae9ae4520fe2e6dc78a766a9ff27"}
Jul 22 18:40:22.817542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.817521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4p574" event={"ID":"89019e31-6d29-49fd-8fc3-f486fd2c1209","Type":"ContainerStarted","Data":"72d462d797e1b22c0ca3ece19f2373beafe2d630ef7147d661a7f12b63f50c05"}
Jul 22 18:40:22.819091 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.819074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l5mbl" event={"ID":"02cd0ca5-45f3-436b-ae62-381b2a755145","Type":"ContainerStarted","Data":"28ce7d9cdacee1016fb88d454e2d9b6345e028bff6273baafabd961a65fe09cc"}
Jul 22 18:40:22.856304 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.856256    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nxss6" podStartSLOduration=7.982189327 podStartE2EDuration="22.856238271s" totalImagesPullingTime="14.874048944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:22.842905309 +0000 UTC m=+23.733446241" watchObservedRunningTime="2026-07-22 18:40:22.856238271 +0000 UTC m=+23.746779203"
Jul 22 18:40:22.856917 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.856440    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4kzc7" podStartSLOduration=2.553722599 podStartE2EDuration="22.856432544s" totalImagesPullingTime="20.302709945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:22.8561517 +0000 UTC m=+23.746692633" watchObservedRunningTime="2026-07-22 18:40:22.856432544 +0000 UTC m=+23.746973476"
Jul 22 18:40:22.869782 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.869739    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4p574" podStartSLOduration=2.5894209569999997 podStartE2EDuration="22.869722765s" totalImagesPullingTime="20.280301808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:22.869412734 +0000 UTC m=+23.759953666" watchObservedRunningTime="2026-07-22 18:40:22.869722765 +0000 UTC m=+23.760263698"
Jul 22 18:40:22.884171 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.884132    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l5mbl" podStartSLOduration=3.841017246 podStartE2EDuration="22.884116011s" totalImagesPullingTime="19.043098765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:22.884077719 +0000 UTC m=+23.774618651" watchObservedRunningTime="2026-07-22 18:40:22.884116011 +0000 UTC m=+23.774656945"
Jul 22 18:40:22.902314 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:22.902272    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wgr2l" podStartSLOduration=2.581099858 podStartE2EDuration="22.902255747s" totalImagesPullingTime="20.321155889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:22.901559819 +0000 UTC m=+23.792100754" watchObservedRunningTime="2026-07-22 18:40:22.902255747 +0000 UTC m=+23.792796682"
Jul 22 18:40:23.582093 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:23.582069    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 18:40:23.682601 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:23.682575    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:23.682784 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:23.682583    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:23.682784 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:23.682612    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:23.682784 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:23.682767    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:23.682942 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:23.682867    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:23.682997 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:23.682937    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:23.824476 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:23.824393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" event={"ID":"bd157c08-d963-4c18-a43d-16418f13baf4","Type":"ContainerStarted","Data":"5d62a0302101f43035363176ce3050eb95310f852fdac4498f4f97908fd62b10"}
Jul 22 18:40:23.826423 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:23.826392    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn" event={"ID":"df97a58c-fc62-4399-aa26-a117c7bd5f5c","Type":"ContainerStarted","Data":"1b5a0e27baa51f617285fa7de0ce92a9e440776e65c42d75b319cece0f9ef780"}
Jul 22 18:40:23.828080 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:23.828052    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-88b78" event={"ID":"be0cb195-07a4-41f1-abb8-80b1bc1a2046","Type":"ContainerStarted","Data":"a05e65baa25d9551f93264f279a2b351862df74d71b4d1af5b03b406f74c274f"}
Jul 22 18:40:24.557789 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:24.557702    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T18:40:23.58209015Z","UUID":"81d5fe06-ec06-4176-83d6-14b1f6b8c2e7","Handler":null,"Name":"","Endpoint":""}
Jul 22 18:40:24.558883 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:24.558864    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 18:40:24.558969 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:24.558899    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 18:40:24.832590 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:24.832526    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn" event={"ID":"df97a58c-fc62-4399-aa26-a117c7bd5f5c","Type":"ContainerStarted","Data":"4e2497f1556f13a0c826924721df1e699e8f6efa09804c000d85addc676e91b0"}
Jul 22 18:40:24.848581 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:24.848542    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwvtn" podStartSLOduration=2.295224856 podStartE2EDuration="24.848527945s" totalImagesPullingTime="22.553303089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:24.848241995 +0000 UTC m=+25.738782927" watchObservedRunningTime="2026-07-22 18:40:24.848527945 +0000 UTC m=+25.739068878"
Jul 22 18:40:24.848767 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:24.848743    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-88b78" podStartSLOduration=4.529854357 podStartE2EDuration="24.848735477s" totalImagesPullingTime="20.31888112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:23.841516923 +0000 UTC m=+24.732057858" watchObservedRunningTime="2026-07-22 18:40:24.848735477 +0000 UTC m=+25.739276464"
Jul 22 18:40:25.682076 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:25.681839    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:25.682350 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:25.681857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:25.682350 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:25.682177    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:25.682350 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:25.682263    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:25.682350 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:25.681908    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:25.682584 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:25.682408    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:25.838821 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:25.838784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" event={"ID":"bd157c08-d963-4c18-a43d-16418f13baf4","Type":"ContainerStarted","Data":"d0c2f06e4cf622e7d21ba233d49248659c302906c8517d6d6cbe9f089f98f2f8"}
Jul 22 18:40:27.371323 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.371143    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:27.373262 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.372152    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:27.681962 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.681800    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:27.682085 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.681805    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:27.682085 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:27.682040    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:27.682165 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:27.682107    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:27.682165 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.681841    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:27.682254 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:27.682236    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:27.848164 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.848128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" event={"ID":"bd157c08-d963-4c18-a43d-16418f13baf4","Type":"ContainerStarted","Data":"a7794980b3afa0539c71f7cf30b9a0221498fdc2b333f911041195f8c0448a92"}
Jul 22 18:40:27.848332 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.848307    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:27.848475 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.848370    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:27.850120 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.850096    2576 generic.go:356] "Generic (PLEG): container finished" podID="f88707f9-0289-4032-a079-321f82e7c4e0" containerID="e4a40375b5702d4dd0ed1cc94883295789e091b2889f49eee7a1e41990b9e908" exitCode=0
Jul 22 18:40:27.850216 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.850186    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qzlbl" event={"ID":"f88707f9-0289-4032-a079-321f82e7c4e0","Type":"ContainerDied","Data":"e4a40375b5702d4dd0ed1cc94883295789e091b2889f49eee7a1e41990b9e908"}
Jul 22 18:40:27.865025 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.865006    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:27.872994 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.872953    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2" podStartSLOduration=7.473211843 podStartE2EDuration="27.87294072s" totalImagesPullingTime="20.399728877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:27.872393112 +0000 UTC m=+28.762934044" watchObservedRunningTime="2026-07-22 18:40:27.87294072 +0000 UTC m=+28.763481651"
Jul 22 18:40:27.912800 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.912783    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:27.913356 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:27.913340    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nxss6"
Jul 22 18:40:28.853045 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:28.853014    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:28.867292 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:28.867271    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:40:29.244953 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:29.244914    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6wvxw"]
Jul 22 18:40:29.245133 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:29.245047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:29.245183 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:29.245159    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:29.245522 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:29.245492    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-shh65"]
Jul 22 18:40:29.245690 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:29.245606    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:29.245773 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:29.245722    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:29.246059 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:29.246035    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xd9sx"]
Jul 22 18:40:29.246156 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:29.246150    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:29.246255 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:29.246238    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:29.856238 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:29.856061    2576 generic.go:356] "Generic (PLEG): container finished" podID="f88707f9-0289-4032-a079-321f82e7c4e0" containerID="d85352bafc588f4ee25779b8f0043ac424437f9a095f604b4cc3754ba99e1b90" exitCode=0
Jul 22 18:40:29.856748 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:29.856145    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qzlbl" event={"ID":"f88707f9-0289-4032-a079-321f82e7c4e0","Type":"ContainerDied","Data":"d85352bafc588f4ee25779b8f0043ac424437f9a095f604b4cc3754ba99e1b90"}
Jul 22 18:40:30.682478 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:30.682446    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:30.682478 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:30.682482    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:30.682699 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:30.682487    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:30.682699 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:30.682567    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:30.682699 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:30.682615    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:30.682812 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:30.682711    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:31.863259 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:31.863231    2576 generic.go:356] "Generic (PLEG): container finished" podID="f88707f9-0289-4032-a079-321f82e7c4e0" containerID="20b1dd0100565cfd6f5889a16867769be3f76ce4408cac4104bbee912ad75de2" exitCode=0
Jul 22 18:40:31.863835 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:31.863309    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qzlbl" event={"ID":"f88707f9-0289-4032-a079-321f82e7c4e0","Type":"ContainerDied","Data":"20b1dd0100565cfd6f5889a16867769be3f76ce4408cac4104bbee912ad75de2"}
Jul 22 18:40:32.682795 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:32.682763    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:32.682795 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:32.682788    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:32.683004 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:32.682763    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:32.683004 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:32.682912    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:32.683081 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:32.683037    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:32.683231 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:32.683111    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:33.237679 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:33.237638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:33.238198 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:33.237779    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:33.238198 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:33.237857    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret podName:04e18ad9-ab77-4ea3-b3b7-3c2a570e096b nodeName:}" failed. No retries permitted until 2026-07-22 18:41:05.237835137 +0000 UTC m=+66.128376050 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret") pod "global-pull-secret-syncer-shh65" (UID: "04e18ad9-ab77-4ea3-b3b7-3c2a570e096b") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:40:33.338635 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:33.338602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:33.338842 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:33.338786    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:40:33.338842 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:33.338812    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:40:33.338842 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:33.338823    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kwj4v for pod openshift-network-diagnostics/network-check-target-6wvxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:33.338954 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:33.338879    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v podName:673d08df-5491-439e-b8de-0a21dd0b72f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:41:05.338860381 +0000 UTC m=+66.229401294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwj4v" (UniqueName: "kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v") pod "network-check-target-6wvxw" (UID: "673d08df-5491-439e-b8de-0a21dd0b72f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:40:33.439257 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:33.439180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:33.439397 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:33.439327    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:33.439397 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:33.439391    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:41:05.439371731 +0000 UTC m=+66.329912647 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:40:34.682436 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:34.682408    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:34.682436 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:34.682434    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:34.682873 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:34.682437    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:34.682873 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:34.682533    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-shh65" podUID="04e18ad9-ab77-4ea3-b3b7-3c2a570e096b"
Jul 22 18:40:34.682873 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:34.682607    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6wvxw" podUID="673d08df-5491-439e-b8de-0a21dd0b72f4"
Jul 22 18:40:34.682873 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:34.682702    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:40:34.976851 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:34.976820    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-185.ec2.internal" event="NodeReady"
Jul 22 18:40:34.977020 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:34.976966    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 18:40:35.018335 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.018274    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vpz2f"]
Jul 22 18:40:35.044907 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.044881    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9h9qz"]
Jul 22 18:40:35.045073 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.044967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.047588 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.047567    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 18:40:35.047740 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.047685    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 18:40:35.047740 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.047691    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hbkg4\""
Jul 22 18:40:35.050802 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.050781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2986a3e4-da73-4650-9389-a4d1a1f22302-tmp-dir\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.050929 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.050811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nllw4\" (UniqueName: \"kubernetes.io/projected/2986a3e4-da73-4650-9389-a4d1a1f22302-kube-api-access-nllw4\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.050992 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.050931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2986a3e4-da73-4650-9389-a4d1a1f22302-config-volume\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.050992 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.050961    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.056847 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.056828    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vpz2f"]
Jul 22 18:40:35.056961 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.056860    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9h9qz"]
Jul 22 18:40:35.056961 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.056918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9h9qz"
Jul 22 18:40:35.059829 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.059803    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 18:40:35.059955 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.059807    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 18:40:35.059955 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.059914    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 18:40:35.059955 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.059928    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zqsx4\""
Jul 22 18:40:35.151502 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.151471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfwq4\" (UniqueName: \"kubernetes.io/projected/5241d666-4a13-4c9d-8ff2-a7448dce1332-kube-api-access-pfwq4\") pod \"ingress-canary-9h9qz\" (UID: \"5241d666-4a13-4c9d-8ff2-a7448dce1332\") " pod="openshift-ingress-canary/ingress-canary-9h9qz"
Jul 22 18:40:35.151675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.151533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2986a3e4-da73-4650-9389-a4d1a1f22302-tmp-dir\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.151675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.151558    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nllw4\" (UniqueName: \"kubernetes.io/projected/2986a3e4-da73-4650-9389-a4d1a1f22302-kube-api-access-nllw4\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.151675 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.151604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5241d666-4a13-4c9d-8ff2-a7448dce1332-cert\") pod \"ingress-canary-9h9qz\" (UID: \"5241d666-4a13-4c9d-8ff2-a7448dce1332\") " pod="openshift-ingress-canary/ingress-canary-9h9qz"
Jul 22 18:40:35.151822 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.151772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2986a3e4-da73-4650-9389-a4d1a1f22302-config-volume\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.151822 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.151801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.151945 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:35.151931    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:40:35.151982 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.151955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2986a3e4-da73-4650-9389-a4d1a1f22302-tmp-dir\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.152020 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:35.152000    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls podName:2986a3e4-da73-4650-9389-a4d1a1f22302 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:35.651980455 +0000 UTC m=+36.542521380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls") pod "dns-default-vpz2f" (UID: "2986a3e4-da73-4650-9389-a4d1a1f22302") : secret "dns-default-metrics-tls" not found
Jul 22 18:40:35.152412 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.152398    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2986a3e4-da73-4650-9389-a4d1a1f22302-config-volume\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.163387 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.163362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nllw4\" (UniqueName: \"kubernetes.io/projected/2986a3e4-da73-4650-9389-a4d1a1f22302-kube-api-access-nllw4\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.252489 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.252403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5241d666-4a13-4c9d-8ff2-a7448dce1332-cert\") pod \"ingress-canary-9h9qz\" (UID: \"5241d666-4a13-4c9d-8ff2-a7448dce1332\") " pod="openshift-ingress-canary/ingress-canary-9h9qz"
Jul 22 18:40:35.252489 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.252476    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfwq4\" (UniqueName: \"kubernetes.io/projected/5241d666-4a13-4c9d-8ff2-a7448dce1332-kube-api-access-pfwq4\") pod \"ingress-canary-9h9qz\" (UID: \"5241d666-4a13-4c9d-8ff2-a7448dce1332\") " pod="openshift-ingress-canary/ingress-canary-9h9qz"
Jul 22 18:40:35.254979 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.254958    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5241d666-4a13-4c9d-8ff2-a7448dce1332-cert\") pod \"ingress-canary-9h9qz\" (UID: \"5241d666-4a13-4c9d-8ff2-a7448dce1332\") " pod="openshift-ingress-canary/ingress-canary-9h9qz"
Jul 22 18:40:35.261023 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.260996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfwq4\" (UniqueName: \"kubernetes.io/projected/5241d666-4a13-4c9d-8ff2-a7448dce1332-kube-api-access-pfwq4\") pod \"ingress-canary-9h9qz\" (UID: \"5241d666-4a13-4c9d-8ff2-a7448dce1332\") " pod="openshift-ingress-canary/ingress-canary-9h9qz"
Jul 22 18:40:35.365981 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.365943    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9h9qz"
Jul 22 18:40:35.655506 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:35.655474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:35.655710 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:35.655599    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:40:35.655777 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:35.655713    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls podName:2986a3e4-da73-4650-9389-a4d1a1f22302 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:36.655694394 +0000 UTC m=+37.546235311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls") pod "dns-default-vpz2f" (UID: "2986a3e4-da73-4650-9389-a4d1a1f22302") : secret "dns-default-metrics-tls" not found
Jul 22 18:40:36.079758 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.079578    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9h9qz"]
Jul 22 18:40:36.153569 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:40:36.153535    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5241d666_4a13_4c9d_8ff2_a7448dce1332.slice/crio-3a323fdf0620a739147853b9c15c354c05925f7d4ac9a2cdeab19c7f2f2a4898 WatchSource:0}: Error finding container 3a323fdf0620a739147853b9c15c354c05925f7d4ac9a2cdeab19c7f2f2a4898: Status 404 returned error can't find the container with id 3a323fdf0620a739147853b9c15c354c05925f7d4ac9a2cdeab19c7f2f2a4898
Jul 22 18:40:36.662664 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.662617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:36.662825 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:36.662771    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:40:36.662866 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:36.662838    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls podName:2986a3e4-da73-4650-9389-a4d1a1f22302 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:38.662821375 +0000 UTC m=+39.553362288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls") pod "dns-default-vpz2f" (UID: "2986a3e4-da73-4650-9389-a4d1a1f22302") : secret "dns-default-metrics-tls" not found
Jul 22 18:40:36.682662 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.682618    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:40:36.682798 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.682621    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:40:36.682850 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.682621    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:40:36.685568 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.685550    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 18:40:36.685723 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.685551    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 18:40:36.686939 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.686918    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-csrvs\""
Jul 22 18:40:36.686982 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.686949    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2vpr\""
Jul 22 18:40:36.687035 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.687012    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 18:40:36.687086 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.687058    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 18:40:36.875994 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.875955    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9h9qz" event={"ID":"5241d666-4a13-4c9d-8ff2-a7448dce1332","Type":"ContainerStarted","Data":"3a323fdf0620a739147853b9c15c354c05925f7d4ac9a2cdeab19c7f2f2a4898"}
Jul 22 18:40:36.878848 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.878821    2576 generic.go:356] "Generic (PLEG): container finished" podID="f88707f9-0289-4032-a079-321f82e7c4e0" containerID="bff5dce699e3fe62bd7a4ce5862e53f0c7f632fe93ee74cda6442710e7f12f83" exitCode=0
Jul 22 18:40:36.878999 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:36.878883    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qzlbl" event={"ID":"f88707f9-0289-4032-a079-321f82e7c4e0","Type":"ContainerDied","Data":"bff5dce699e3fe62bd7a4ce5862e53f0c7f632fe93ee74cda6442710e7f12f83"}
Jul 22 18:40:37.884572 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:37.884538    2576 generic.go:356] "Generic (PLEG): container finished" podID="f88707f9-0289-4032-a079-321f82e7c4e0" containerID="b8a13a0255a58e413a81941bccd36d92777ee952ea5e78b7f2b0aac1773388aa" exitCode=0
Jul 22 18:40:37.885007 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:37.884613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qzlbl" event={"ID":"f88707f9-0289-4032-a079-321f82e7c4e0","Type":"ContainerDied","Data":"b8a13a0255a58e413a81941bccd36d92777ee952ea5e78b7f2b0aac1773388aa"}
Jul 22 18:40:38.679158 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:38.678904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:38.679158 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:38.679049    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:40:38.679383 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:38.679214    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls podName:2986a3e4-da73-4650-9389-a4d1a1f22302 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:42.67919924 +0000 UTC m=+43.569740170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls") pod "dns-default-vpz2f" (UID: "2986a3e4-da73-4650-9389-a4d1a1f22302") : secret "dns-default-metrics-tls" not found
Jul 22 18:40:38.887886 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:38.887852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9h9qz" event={"ID":"5241d666-4a13-4c9d-8ff2-a7448dce1332","Type":"ContainerStarted","Data":"6a75f54982b34b851ab1e20db7becb09dc2883681eb0581a1780770770f5b24e"}
Jul 22 18:40:38.891060 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:38.891034    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qzlbl" event={"ID":"f88707f9-0289-4032-a079-321f82e7c4e0","Type":"ContainerStarted","Data":"092b2209c129f8e98a8136d011bdc35b05a72f5ff140f3531b06055c3eaf7d36"}
Jul 22 18:40:38.903311 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:38.903270    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9h9qz" podStartSLOduration=1.624906015 podStartE2EDuration="3.903255644s" totalImagesPullingTime="2.278349629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:38.902709439 +0000 UTC m=+39.793250371" watchObservedRunningTime="2026-07-22 18:40:38.903255644 +0000 UTC m=+39.793796554"
Jul 22 18:40:38.925621 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:38.925580    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qzlbl" podStartSLOduration=7.918527589 podStartE2EDuration="38.925567684s" totalImagesPullingTime="31.007040095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:38.92421365 +0000 UTC m=+39.814754584" watchObservedRunningTime="2026-07-22 18:40:38.925567684 +0000 UTC m=+39.816108615"
Jul 22 18:40:42.705450 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:42.705414    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:42.705909 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:42.705562    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:40:42.705909 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:42.705642    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls podName:2986a3e4-da73-4650-9389-a4d1a1f22302 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:50.705609189 +0000 UTC m=+51.596150098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls") pod "dns-default-vpz2f" (UID: "2986a3e4-da73-4650-9389-a4d1a1f22302") : secret "dns-default-metrics-tls" not found
Jul 22 18:40:50.760873 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:40:50.760839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:40:50.761320 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:50.760963    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:40:50.761320 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:40:50.761031    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls podName:2986a3e4-da73-4650-9389-a4d1a1f22302 nodeName:}" failed. No retries permitted until 2026-07-22 18:41:06.761011896 +0000 UTC m=+67.651552826 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls") pod "dns-default-vpz2f" (UID: "2986a3e4-da73-4650-9389-a4d1a1f22302") : secret "dns-default-metrics-tls" not found
Jul 22 18:41:00.868857 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:00.868825    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c88z2"
Jul 22 18:41:05.256408 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.256367    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:41:05.259117 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.259100    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 18:41:05.270905 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.270885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/04e18ad9-ab77-4ea3-b3b7-3c2a570e096b-original-pull-secret\") pod \"global-pull-secret-syncer-shh65\" (UID: \"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b\") " pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:41:05.357364 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.357331    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:41:05.360191 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.360174    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 18:41:05.370657 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.370638    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 18:41:05.380917 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.380893    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwj4v\" (UniqueName: \"kubernetes.io/projected/673d08df-5491-439e-b8de-0a21dd0b72f4-kube-api-access-kwj4v\") pod \"network-check-target-6wvxw\" (UID: \"673d08df-5491-439e-b8de-0a21dd0b72f4\") " pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:41:05.458036 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.458010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:41:05.461749 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.461732    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 18:41:05.469232 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:41:05.469218    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 18:41:05.469298 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:41:05.469280    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:42:09.469264772 +0000 UTC m=+130.359805682 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : secret "metrics-daemon-secret" not found
Jul 22 18:41:05.492597 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.492573    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-shh65"
Jul 22 18:41:05.500711 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.500690    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-csrvs\""
Jul 22 18:41:05.509049 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.509003    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:41:05.618125 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.618101    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-shh65"]
Jul 22 18:41:05.633017 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.632937    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6wvxw"]
Jul 22 18:41:05.636516 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:41:05.636492    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673d08df_5491_439e_b8de_0a21dd0b72f4.slice/crio-caeb2e8fe2dad8d951a18ccab92be25b23a953c284a1071491e66b4daab50c21 WatchSource:0}: Error finding container caeb2e8fe2dad8d951a18ccab92be25b23a953c284a1071491e66b4daab50c21: Status 404 returned error can't find the container with id caeb2e8fe2dad8d951a18ccab92be25b23a953c284a1071491e66b4daab50c21
Jul 22 18:41:05.948641 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.948593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6wvxw" event={"ID":"673d08df-5491-439e-b8de-0a21dd0b72f4","Type":"ContainerStarted","Data":"caeb2e8fe2dad8d951a18ccab92be25b23a953c284a1071491e66b4daab50c21"}
Jul 22 18:41:05.949494 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:05.949473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-shh65" event={"ID":"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b","Type":"ContainerStarted","Data":"ffb475af27b780dc2ad5edcf09a64465c888729a6821be31c878521561fb0b8f"}
Jul 22 18:41:06.767602 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:06.767371    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:41:06.767602 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:41:06.767546    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:41:06.767602 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:41:06.767611    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls podName:2986a3e4-da73-4650-9389-a4d1a1f22302 nodeName:}" failed. No retries permitted until 2026-07-22 18:41:38.767590685 +0000 UTC m=+99.658131602 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls") pod "dns-default-vpz2f" (UID: "2986a3e4-da73-4650-9389-a4d1a1f22302") : secret "dns-default-metrics-tls" not found
Jul 22 18:41:10.963386 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:10.963340    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6wvxw" event={"ID":"673d08df-5491-439e-b8de-0a21dd0b72f4","Type":"ContainerStarted","Data":"f2784d210a0a52072f9cd6d422ee904e375596ff29409387baa3f6a195aeca7d"}
Jul 22 18:41:10.963822 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:10.963442    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:41:10.964797 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:10.964756    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-shh65" event={"ID":"04e18ad9-ab77-4ea3-b3b7-3c2a570e096b","Type":"ContainerStarted","Data":"5e7d185fb9b3cc2885ae2c260576b904c9f3e0349cd4c42e673eb5bfd5a71f15"}
Jul 22 18:41:10.992078 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:10.992030    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6wvxw" podStartSLOduration=66.474550781 podStartE2EDuration="1m10.992019027s" totalImagesPullingTime="4.517468246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:41:10.978370825 +0000 UTC m=+71.868911754" watchObservedRunningTime="2026-07-22 18:41:10.992019027 +0000 UTC m=+71.882559959"
Jul 22 18:41:10.992272 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:10.992252    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-shh65" podStartSLOduration=66.45015818 podStartE2EDuration="1m10.992247323s" totalImagesPullingTime="4.542089143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:41:10.991620211 +0000 UTC m=+71.882161144" watchObservedRunningTime="2026-07-22 18:41:10.992247323 +0000 UTC m=+71.882788310"
Jul 22 18:41:17.733927 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.733891    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"]
Jul 22 18:41:17.735829 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.735814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"
Jul 22 18:41:17.737356 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.737326    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"]
Jul 22 18:41:17.739117 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.739097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.739515 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.739482    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-q9c9d\""
Jul 22 18:41:17.739515 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.739496    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 22 18:41:17.739515 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.739509    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 22 18:41:17.739770 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.739494    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 22 18:41:17.739770 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.739489    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 22 18:41:17.741387 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.741346    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 22 18:41:17.741674 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.741654    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 22 18:41:17.741758 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.741742    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 22 18:41:17.741950 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.741936    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 22 18:41:17.747322 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.747303    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"]
Jul 22 18:41:17.762180 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.762153    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"]
Jul 22 18:41:17.844179 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.844156    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4ngn\" (UniqueName: \"kubernetes.io/projected/372ec1f1-b642-4831-bb63-cb9b0a89d00a-kube-api-access-j4ngn\") pod \"managed-serviceaccount-addon-agent-6bf9b59868-zch6x\" (UID: \"372ec1f1-b642-4831-bb63-cb9b0a89d00a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"
Jul 22 18:41:17.844274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.844185    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-hub\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.844274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.844203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.844274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.844221    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.844274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.844252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8h9n\" (UniqueName: \"kubernetes.io/projected/d92e7e20-9642-473e-9903-b29b4bcfc456-kube-api-access-v8h9n\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.844403 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.844355    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d92e7e20-9642-473e-9903-b29b4bcfc456-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.844403 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.844392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-ca\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.844472 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.844446    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/372ec1f1-b642-4831-bb63-cb9b0a89d00a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bf9b59868-zch6x\" (UID: \"372ec1f1-b642-4831-bb63-cb9b0a89d00a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"
Jul 22 18:41:17.945681 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.945654    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d92e7e20-9642-473e-9903-b29b4bcfc456-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.945781 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.945690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-ca\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.945781 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.945727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/372ec1f1-b642-4831-bb63-cb9b0a89d00a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bf9b59868-zch6x\" (UID: \"372ec1f1-b642-4831-bb63-cb9b0a89d00a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"
Jul 22 18:41:17.945781 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.945745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4ngn\" (UniqueName: \"kubernetes.io/projected/372ec1f1-b642-4831-bb63-cb9b0a89d00a-kube-api-access-j4ngn\") pod \"managed-serviceaccount-addon-agent-6bf9b59868-zch6x\" (UID: \"372ec1f1-b642-4831-bb63-cb9b0a89d00a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"
Jul 22 18:41:17.945781 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.945763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-hub\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.945781 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.945777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.946011 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.945794    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.946011 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.945812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8h9n\" (UniqueName: \"kubernetes.io/projected/d92e7e20-9642-473e-9903-b29b4bcfc456-kube-api-access-v8h9n\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.946437 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.946411    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d92e7e20-9642-473e-9903-b29b4bcfc456-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.948139 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.948110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.948440 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.948409    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-hub\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.948440 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.948437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/372ec1f1-b642-4831-bb63-cb9b0a89d00a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bf9b59868-zch6x\" (UID: \"372ec1f1-b642-4831-bb63-cb9b0a89d00a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"
Jul 22 18:41:17.948575 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.948446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-ca\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.948873 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.948852    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d92e7e20-9642-473e-9903-b29b4bcfc456-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.953859 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.953836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8h9n\" (UniqueName: \"kubernetes.io/projected/d92e7e20-9642-473e-9903-b29b4bcfc456-kube-api-access-v8h9n\") pod \"cluster-proxy-proxy-agent-7b889d84cd-s8m5j\" (UID: \"d92e7e20-9642-473e-9903-b29b4bcfc456\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:17.953934 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:17.953863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4ngn\" (UniqueName: \"kubernetes.io/projected/372ec1f1-b642-4831-bb63-cb9b0a89d00a-kube-api-access-j4ngn\") pod \"managed-serviceaccount-addon-agent-6bf9b59868-zch6x\" (UID: \"372ec1f1-b642-4831-bb63-cb9b0a89d00a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"
Jul 22 18:41:18.063780 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:18.063695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"
Jul 22 18:41:18.068164 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:18.068138    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:41:18.183738 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:18.183710    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x"]
Jul 22 18:41:18.186897 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:41:18.186872    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod372ec1f1_b642_4831_bb63_cb9b0a89d00a.slice/crio-b3ed5f56d2d1c74c61b12f8b26a65d1936a9a83b57ee6772cf0dbef9a6bfe856 WatchSource:0}: Error finding container b3ed5f56d2d1c74c61b12f8b26a65d1936a9a83b57ee6772cf0dbef9a6bfe856: Status 404 returned error can't find the container with id b3ed5f56d2d1c74c61b12f8b26a65d1936a9a83b57ee6772cf0dbef9a6bfe856
Jul 22 18:41:18.212452 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:18.212430    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"]
Jul 22 18:41:18.215479 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:41:18.215457    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd92e7e20_9642_473e_9903_b29b4bcfc456.slice/crio-04b06a4b7b75ead1d11e82f164604568adb263f00b7294ee86035553224dfb37 WatchSource:0}: Error finding container 04b06a4b7b75ead1d11e82f164604568adb263f00b7294ee86035553224dfb37: Status 404 returned error can't find the container with id 04b06a4b7b75ead1d11e82f164604568adb263f00b7294ee86035553224dfb37
Jul 22 18:41:18.984208 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:18.984174    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x" event={"ID":"372ec1f1-b642-4831-bb63-cb9b0a89d00a","Type":"ContainerStarted","Data":"b3ed5f56d2d1c74c61b12f8b26a65d1936a9a83b57ee6772cf0dbef9a6bfe856"}
Jul 22 18:41:18.985213 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:18.985191    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" event={"ID":"d92e7e20-9642-473e-9903-b29b4bcfc456","Type":"ContainerStarted","Data":"04b06a4b7b75ead1d11e82f164604568adb263f00b7294ee86035553224dfb37"}
Jul 22 18:41:21.995483 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:21.995325    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x" event={"ID":"372ec1f1-b642-4831-bb63-cb9b0a89d00a","Type":"ContainerStarted","Data":"b9f76c525fa523e4cb46df88f3e5615b2558f2f3d72a71daf5cc7efcc2a1de7d"}
Jul 22 18:41:22.010472 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:22.010431    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x" podStartSLOduration=2.019046231 podStartE2EDuration="5.010418398s" totalImagesPullingTime="2.991372167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:41:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:41:22.009030112 +0000 UTC m=+82.899571045" watchObservedRunningTime="2026-07-22 18:41:22.010418398 +0000 UTC m=+82.900959363"
Jul 22 18:41:23.001202 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:23.001164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" event={"ID":"d92e7e20-9642-473e-9903-b29b4bcfc456","Type":"ContainerStarted","Data":"ca1244ca0a9cbc17bf93e24d9f49c52b328cfc0c283b0d52e8034a0ef4cddc5f"}
Jul 22 18:41:23.001711 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:23.001214    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" event={"ID":"d92e7e20-9642-473e-9903-b29b4bcfc456","Type":"ContainerStarted","Data":"80a3b5e59e7e333dc07f3737d8b8b66561c8d9bb24a471528c0dd1616d41219c"}
Jul 22 18:41:23.001711 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:23.001233    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" event={"ID":"d92e7e20-9642-473e-9903-b29b4bcfc456","Type":"ContainerStarted","Data":"c74c9163e843cdc19d2d5a059b8746c9b70151339c5b10c997eb1481e35439c4"}
Jul 22 18:41:23.020017 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:23.019974    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" podStartSLOduration=1.727048168 podStartE2EDuration="6.019959591s" totalImagesPullingTime="4.292911423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:41:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:41:23.019619375 +0000 UTC m=+83.910160320" watchObservedRunningTime="2026-07-22 18:41:23.019959591 +0000 UTC m=+83.910500524"
Jul 22 18:41:38.792241 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:38.792206    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:41:38.792781 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:41:38.792324    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:41:38.792781 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:41:38.792376    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls podName:2986a3e4-da73-4650-9389-a4d1a1f22302 nodeName:}" failed. No retries permitted until 2026-07-22 18:42:42.792362454 +0000 UTC m=+163.682903364 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls") pod "dns-default-vpz2f" (UID: "2986a3e4-da73-4650-9389-a4d1a1f22302") : secret "dns-default-metrics-tls" not found
Jul 22 18:41:41.969670 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:41:41.969619    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6wvxw"
Jul 22 18:42:09.503033 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:42:09.503003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:42:09.503490 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:42:09.503105    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 18:42:09.503490 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:42:09.503157    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:11.503143058 +0000 UTC m=+252.393683968 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : secret "metrics-daemon-secret" not found
Jul 22 18:42:38.055382 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:42:38.055334    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vpz2f" podUID="2986a3e4-da73-4650-9389-a4d1a1f22302"
Jul 22 18:42:38.212003 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:42:38.211973    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:42:39.702089 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:42:39.702047    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:42:42.826971 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:42:42.826932    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:42:42.827343 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:42:42.827077    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:42:42.827343 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:42:42.827143    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls podName:2986a3e4-da73-4650-9389-a4d1a1f22302 nodeName:}" failed. No retries permitted until 2026-07-22 18:44:44.827124486 +0000 UTC m=+285.717665396 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls") pod "dns-default-vpz2f" (UID: "2986a3e4-da73-4650-9389-a4d1a1f22302") : secret "dns-default-metrics-tls" not found
Jul 22 18:42:53.682682 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:42:53.682583    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:43:22.333588 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:43:22.333553    2576 generic.go:356] "Generic (PLEG): container finished" podID="372ec1f1-b642-4831-bb63-cb9b0a89d00a" containerID="b9f76c525fa523e4cb46df88f3e5615b2558f2f3d72a71daf5cc7efcc2a1de7d" exitCode=255
Jul 22 18:43:22.334106 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:43:22.333643    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x" event={"ID":"372ec1f1-b642-4831-bb63-cb9b0a89d00a","Type":"ContainerDied","Data":"b9f76c525fa523e4cb46df88f3e5615b2558f2f3d72a71daf5cc7efcc2a1de7d"}
Jul 22 18:43:22.334106 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:43:22.334012    2576 scope.go:117] "RemoveContainer" containerID="b9f76c525fa523e4cb46df88f3e5615b2558f2f3d72a71daf5cc7efcc2a1de7d"
Jul 22 18:43:23.338407 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:43:23.338371    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf9b59868-zch6x" event={"ID":"372ec1f1-b642-4831-bb63-cb9b0a89d00a","Type":"ContainerStarted","Data":"6484405568b6ba70685f01dd4a967d5e7865bab70a1f09d0ee490ebccae2ff15"}
Jul 22 18:44:11.591835 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:11.591797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:44:11.592412 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:11.592280    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 18:44:11.592475 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:11.592441    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs podName:881e051a-b4aa-4d47-b389-0f3dd300de2f nodeName:}" failed. No retries permitted until 2026-07-22 18:46:13.592401354 +0000 UTC m=+374.482942277 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs") pod "network-metrics-daemon-xd9sx" (UID: "881e051a-b4aa-4d47-b389-0f3dd300de2f") : secret "metrics-daemon-secret" not found
Jul 22 18:44:25.298800 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.298725    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cg27c"]
Jul 22 18:44:25.301266 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.301250    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.303767 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.303742    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 18:44:25.303894 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.303867    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 18:44:25.304191 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.304171    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 18:44:25.304258 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.304197    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 18:44:25.305433 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.305413    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rvpd8\""
Jul 22 18:44:25.310954 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.310934    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cg27c"]
Jul 22 18:44:25.487052 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.487012    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70d227ba-ecb7-446b-a997-e770e52d894d-crio-socket\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.487052 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.487055    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs96f\" (UniqueName: \"kubernetes.io/projected/70d227ba-ecb7-446b-a997-e770e52d894d-kube-api-access-vs96f\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.487246 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.487096    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.487246 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.487134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.487246 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.487192    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70d227ba-ecb7-446b-a997-e770e52d894d-data-volume\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.588144 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.588073    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70d227ba-ecb7-446b-a997-e770e52d894d-data-volume\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.588144 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.588110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70d227ba-ecb7-446b-a997-e770e52d894d-crio-socket\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.588144 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.588127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs96f\" (UniqueName: \"kubernetes.io/projected/70d227ba-ecb7-446b-a997-e770e52d894d-kube-api-access-vs96f\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.588340 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.588146    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.588340 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.588168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.588340 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.588207    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70d227ba-ecb7-446b-a997-e770e52d894d-crio-socket\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.588340 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:25.588229    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:44:25.588340 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:25.588253    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:25.588340 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:25.588301    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:26.088281137 +0000 UTC m=+266.978822104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : configmap "kube-rbac-proxy" not found
Jul 22 18:44:25.588340 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:25.588316    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:26.088309758 +0000 UTC m=+266.978850668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:25.588574 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.588387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70d227ba-ecb7-446b-a997-e770e52d894d-data-volume\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:25.600711 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:25.600689    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs96f\" (UniqueName: \"kubernetes.io/projected/70d227ba-ecb7-446b-a997-e770e52d894d-kube-api-access-vs96f\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:26.091012 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:26.090969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:26.091012 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:26.091015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:26.091218 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:26.091085    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:44:26.091218 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:26.091105    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:26.091218 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:26.091153    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:27.091133299 +0000 UTC m=+267.981674210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : configmap "kube-rbac-proxy" not found
Jul 22 18:44:26.091218 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:26.091167    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:27.091161646 +0000 UTC m=+267.981702556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:27.098216 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:27.098182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:27.098680 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:27.098221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:27.098680 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:27.098326    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:44:27.098680 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:27.098335    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:27.098680 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:27.098426    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:29.098403322 +0000 UTC m=+269.988944232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : configmap "kube-rbac-proxy" not found
Jul 22 18:44:27.098680 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:27.098462    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:29.098450111 +0000 UTC m=+269.988991022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:27.883568 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:27.883538    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4kzc7_ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab/dns-node-resolver/0.log"
Jul 22 18:44:28.884643 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:28.884607    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4p574_89019e31-6d29-49fd-8fc3-f486fd2c1209/node-ca/0.log"
Jul 22 18:44:29.115790 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:29.115747    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:29.115956 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:29.115848    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:29.115956 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:29.115899    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:29.115956 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:29.115909    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:44:29.116050 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:29.115967    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:33.115950012 +0000 UTC m=+274.006490922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:29.116050 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:29.115981    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:33.115974788 +0000 UTC m=+274.006515698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : configmap "kube-rbac-proxy" not found
Jul 22 18:44:29.883832 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:29.883803    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9h9qz_5241d666-4a13-4c9d-8ff2-a7448dce1332/serve-healthcheck-canary/0.log"
Jul 22 18:44:33.144188 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:33.144149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:33.144188 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:33.144194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:33.144736 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:33.144259    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:44:33.144736 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:33.144329    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:33.144736 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:33.144332    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:41.144316253 +0000 UTC m=+282.034857166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : configmap "kube-rbac-proxy" not found
Jul 22 18:44:33.144736 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:33.144384    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:41.144373077 +0000 UTC m=+282.034913986 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:41.198262 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:41.198224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:41.198262 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:41.198262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:41.198709 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:41.198370    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:41.198709 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:41.198445    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls podName:70d227ba-ecb7-446b-a997-e770e52d894d nodeName:}" failed. No retries permitted until 2026-07-22 18:44:57.198427016 +0000 UTC m=+298.088967926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cg27c" (UID: "70d227ba-ecb7-446b-a997-e770e52d894d") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:44:41.198906 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:41.198887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70d227ba-ecb7-446b-a997-e770e52d894d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:41.212574 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:41.212549    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vpz2f" podUID="2986a3e4-da73-4650-9389-a4d1a1f22302"
Jul 22 18:44:41.552743 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:41.552661    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:44:44.922662 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:44.922606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:44:44.924879 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:44.924860    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2986a3e4-da73-4650-9389-a4d1a1f22302-metrics-tls\") pod \"dns-default-vpz2f\" (UID: \"2986a3e4-da73-4650-9389-a4d1a1f22302\") " pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:44:45.157912 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:45.157880    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hbkg4\""
Jul 22 18:44:45.163658 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:45.163641    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:44:45.282212 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:45.282129    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vpz2f"]
Jul 22 18:44:45.284862 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:44:45.284832    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2986a3e4_da73_4650_9389_a4d1a1f22302.slice/crio-360091b3ba48747914f3cd58b9cd5da6d4801a4e1ac26640128efc680bd8f1dc WatchSource:0}: Error finding container 360091b3ba48747914f3cd58b9cd5da6d4801a4e1ac26640128efc680bd8f1dc: Status 404 returned error can't find the container with id 360091b3ba48747914f3cd58b9cd5da6d4801a4e1ac26640128efc680bd8f1dc
Jul 22 18:44:45.566918 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:45.566834    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vpz2f" event={"ID":"2986a3e4-da73-4650-9389-a4d1a1f22302","Type":"ContainerStarted","Data":"360091b3ba48747914f3cd58b9cd5da6d4801a4e1ac26640128efc680bd8f1dc"}
Jul 22 18:44:47.573532 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.573503    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vpz2f" event={"ID":"2986a3e4-da73-4650-9389-a4d1a1f22302","Type":"ContainerStarted","Data":"c3a4106d2da0e7b83e7bfc0f6a8ebc507bc045705b6076838d1d57902c0ac27e"}
Jul 22 18:44:47.573857 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.573554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vpz2f" event={"ID":"2986a3e4-da73-4650-9389-a4d1a1f22302","Type":"ContainerStarted","Data":"7d5745b1547190df9faafb5ee34efc6c1c9769fc80aae2ed0b51e1a257dd96c3"}
Jul 22 18:44:47.573857 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.573656    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:44:47.591445 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.591403    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vpz2f" podStartSLOduration=250.510745513 podStartE2EDuration="4m12.591389395s" totalImagesPullingTime="2.080643882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:47.589709233 +0000 UTC m=+288.480250165" watchObservedRunningTime="2026-07-22 18:44:47.591389395 +0000 UTC m=+288.481930326"
Jul 22 18:44:47.987504 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.987471    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65d7fc9fc6-j56rc"]
Jul 22 18:44:47.990285 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.990269    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:47.993183 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.993158    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 18:44:47.993307 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.993205    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bzh2s\""
Jul 22 18:44:47.993478 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.993460    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 18:44:47.994508 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.994486    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 18:44:47.999525 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:47.999507    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 18:44:48.003232 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.003211    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65d7fc9fc6-j56rc"]
Jul 22 18:44:48.044701 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.044678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8be5a1ca-3804-49cc-8ce9-5daae8826765-registry-tls\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.044794 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.044707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8be5a1ca-3804-49cc-8ce9-5daae8826765-registry-certificates\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.044794 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.044738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8be5a1ca-3804-49cc-8ce9-5daae8826765-installation-pull-secrets\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.044866 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.044789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8be5a1ca-3804-49cc-8ce9-5daae8826765-bound-sa-token\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.044866 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.044830    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlhnc\" (UniqueName: \"kubernetes.io/projected/8be5a1ca-3804-49cc-8ce9-5daae8826765-kube-api-access-vlhnc\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.044866 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.044854    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8be5a1ca-3804-49cc-8ce9-5daae8826765-image-registry-private-configuration\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.044966 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.044880    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8be5a1ca-3804-49cc-8ce9-5daae8826765-ca-trust-extracted\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.044966 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.044898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8be5a1ca-3804-49cc-8ce9-5daae8826765-trusted-ca\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.146020 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.145994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8be5a1ca-3804-49cc-8ce9-5daae8826765-installation-pull-secrets\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.146128 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.146028    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8be5a1ca-3804-49cc-8ce9-5daae8826765-bound-sa-token\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.146128 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.146066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlhnc\" (UniqueName: \"kubernetes.io/projected/8be5a1ca-3804-49cc-8ce9-5daae8826765-kube-api-access-vlhnc\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.146128 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.146096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8be5a1ca-3804-49cc-8ce9-5daae8826765-image-registry-private-configuration\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.146282 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.146131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8be5a1ca-3804-49cc-8ce9-5daae8826765-ca-trust-extracted\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.146282 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.146152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8be5a1ca-3804-49cc-8ce9-5daae8826765-trusted-ca\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.146282 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.146177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8be5a1ca-3804-49cc-8ce9-5daae8826765-registry-tls\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.146282 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.146206    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8be5a1ca-3804-49cc-8ce9-5daae8826765-registry-certificates\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.146728 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.146708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8be5a1ca-3804-49cc-8ce9-5daae8826765-ca-trust-extracted\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.147141 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.147116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8be5a1ca-3804-49cc-8ce9-5daae8826765-registry-certificates\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.147303 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.147278    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8be5a1ca-3804-49cc-8ce9-5daae8826765-trusted-ca\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.148574 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.148556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8be5a1ca-3804-49cc-8ce9-5daae8826765-installation-pull-secrets\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.148667 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.148604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8be5a1ca-3804-49cc-8ce9-5daae8826765-image-registry-private-configuration\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.148855 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.148834    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8be5a1ca-3804-49cc-8ce9-5daae8826765-registry-tls\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.157553 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.157534    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8be5a1ca-3804-49cc-8ce9-5daae8826765-bound-sa-token\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.160101 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.160082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlhnc\" (UniqueName: \"kubernetes.io/projected/8be5a1ca-3804-49cc-8ce9-5daae8826765-kube-api-access-vlhnc\") pod \"image-registry-65d7fc9fc6-j56rc\" (UID: \"8be5a1ca-3804-49cc-8ce9-5daae8826765\") " pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.303096 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.303034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:48.425916 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.425891    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65d7fc9fc6-j56rc"]
Jul 22 18:44:48.428810 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:44:48.428780    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be5a1ca_3804_49cc_8ce9_5daae8826765.slice/crio-fb3f04082c4b709cda7d6f7c5f5d7bb3dd191ecb251541aeab2e1ce8bcd01ce5 WatchSource:0}: Error finding container fb3f04082c4b709cda7d6f7c5f5d7bb3dd191ecb251541aeab2e1ce8bcd01ce5: Status 404 returned error can't find the container with id fb3f04082c4b709cda7d6f7c5f5d7bb3dd191ecb251541aeab2e1ce8bcd01ce5
Jul 22 18:44:48.578407 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.578324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc" event={"ID":"8be5a1ca-3804-49cc-8ce9-5daae8826765","Type":"ContainerStarted","Data":"48124f307debfc09f1b4dca2236d91027543a370e47470a53fedc42186e89421"}
Jul 22 18:44:48.578407 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.578385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc" event={"ID":"8be5a1ca-3804-49cc-8ce9-5daae8826765","Type":"ContainerStarted","Data":"fb3f04082c4b709cda7d6f7c5f5d7bb3dd191ecb251541aeab2e1ce8bcd01ce5"}
Jul 22 18:44:48.597603 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:48.597555    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc" podStartSLOduration=1.597542239 podStartE2EDuration="1.597542239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:48.596653028 +0000 UTC m=+289.487193960" watchObservedRunningTime="2026-07-22 18:44:48.597542239 +0000 UTC m=+289.488083204"
Jul 22 18:44:49.581876 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:49.581846    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:44:56.683678 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:44:56.683612    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xd9sx" podUID="881e051a-b4aa-4d47-b389-0f3dd300de2f"
Jul 22 18:44:57.209737 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:57.209704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:57.211879 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:57.211860    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70d227ba-ecb7-446b-a997-e770e52d894d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cg27c\" (UID: \"70d227ba-ecb7-446b-a997-e770e52d894d\") " pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:57.410187 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:57.410134    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cg27c"
Jul 22 18:44:57.530027 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:57.529999    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cg27c"]
Jul 22 18:44:57.533130 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:44:57.533099    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70d227ba_ecb7_446b_a997_e770e52d894d.slice/crio-af1e7a57cf6818c36ad58622900427c03ef32e0bcd3e39ccf1f6083075bbfff5 WatchSource:0}: Error finding container af1e7a57cf6818c36ad58622900427c03ef32e0bcd3e39ccf1f6083075bbfff5: Status 404 returned error can't find the container with id af1e7a57cf6818c36ad58622900427c03ef32e0bcd3e39ccf1f6083075bbfff5
Jul 22 18:44:57.580026 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:57.580003    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vpz2f"
Jul 22 18:44:57.606268 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:57.606241    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cg27c" event={"ID":"70d227ba-ecb7-446b-a997-e770e52d894d","Type":"ContainerStarted","Data":"af1e7a57cf6818c36ad58622900427c03ef32e0bcd3e39ccf1f6083075bbfff5"}
Jul 22 18:44:58.070017 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:58.069976    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" podUID="d92e7e20-9642-473e-9903-b29b4bcfc456" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 22 18:44:59.461387 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:44:59.461356    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 18:45:04.294426 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:04.294400    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 18:45:04.630732 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:04.630701    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cg27c" event={"ID":"70d227ba-ecb7-446b-a997-e770e52d894d","Type":"ContainerStarted","Data":"f019f6ef84c9df40773347b2cdf43730bb1619677622e86fb9c6a879db38bb4b"}
Jul 22 18:45:08.069776 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:08.069697    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" podUID="d92e7e20-9642-473e-9903-b29b4bcfc456" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 22 18:45:08.645788 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:08.645754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cg27c" event={"ID":"70d227ba-ecb7-446b-a997-e770e52d894d","Type":"ContainerStarted","Data":"e01898da7fc20c9361db3dcb052606f61ace00dbbf82fa2637ba2643807f87a8"}
Jul 22 18:45:09.198423 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.198391    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"]
Jul 22 18:45:09.201479 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.201461    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.205699 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.205622    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 18:45:09.205699 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.205649    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 22 18:45:09.205699 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.205660    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 22 18:45:09.205699 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.205679    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 18:45:09.205928 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.205707    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 18:45:09.205928 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.205679    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nxjnx\""
Jul 22 18:45:09.213862 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.213841    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"]
Jul 22 18:45:09.244399 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.244379    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vlm28"]
Jul 22 18:45:09.247227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.247212    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.250299 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.250279    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 18:45:09.250299 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.250296    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 18:45:09.250446 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.250302    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bns8p\""
Jul 22 18:45:09.250446 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.250329    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 18:45:09.297869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.297847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-842tk\" (UniqueName: \"kubernetes.io/projected/6ba43672-693f-40b1-831a-545fc97ad5f4-kube-api-access-842tk\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.297993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.297883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fe71b81-c0de-4ea0-92a5-332e60d78b99-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.297993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.297909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6ba43672-693f-40b1-831a-545fc97ad5f4-root\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.297993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.297930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-textfile\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.297993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.297962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3fe71b81-c0de-4ea0-92a5-332e60d78b99-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.297993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.297983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-tls\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.298274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.298009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ba43672-693f-40b1-831a-545fc97ad5f4-sys\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.298274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.298054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.298274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.298083    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ghfw\" (UniqueName: \"kubernetes.io/projected/3fe71b81-c0de-4ea0-92a5-332e60d78b99-kube-api-access-2ghfw\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.298274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.298112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fe71b81-c0de-4ea0-92a5-332e60d78b99-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.298274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.298201    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-wtmp\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.298274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.298237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ba43672-693f-40b1-831a-545fc97ad5f4-metrics-client-ca\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.298462 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.298276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399198 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-wtmp\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399326 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399204    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ba43672-693f-40b1-831a-545fc97ad5f4-metrics-client-ca\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399326 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399326 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-842tk\" (UniqueName: \"kubernetes.io/projected/6ba43672-693f-40b1-831a-545fc97ad5f4-kube-api-access-842tk\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399326 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399254    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fe71b81-c0de-4ea0-92a5-332e60d78b99-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.399542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399327    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-wtmp\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399521    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6ba43672-693f-40b1-831a-545fc97ad5f4-root\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399861 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6ba43672-693f-40b1-831a-545fc97ad5f4-root\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399961 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-textfile\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399961 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399913    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ba43672-693f-40b1-831a-545fc97ad5f4-metrics-client-ca\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.399961 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399920    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3fe71b81-c0de-4ea0-92a5-332e60d78b99-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.399961 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-tls\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.400162 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.399989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ba43672-693f-40b1-831a-545fc97ad5f4-sys\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.400162 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.400073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fe71b81-c0de-4ea0-92a5-332e60d78b99-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.400162 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.400087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ba43672-693f-40b1-831a-545fc97ad5f4-sys\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.400162 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.400127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.400355 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.400176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-textfile\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.400355 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:45:09.400334    2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 22 18:45:09.400458 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:45:09.400397    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe71b81-c0de-4ea0-92a5-332e60d78b99-openshift-state-metrics-tls podName:3fe71b81-c0de-4ea0-92a5-332e60d78b99 nodeName:}" failed. No retries permitted until 2026-07-22 18:45:09.900377283 +0000 UTC m=+310.790918197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3fe71b81-c0de-4ea0-92a5-332e60d78b99-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-htcvp" (UID: "3fe71b81-c0de-4ea0-92a5-332e60d78b99") : secret "openshift-state-metrics-tls" not found
Jul 22 18:45:09.400679 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.400656    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.400777 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.400157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ghfw\" (UniqueName: \"kubernetes.io/projected/3fe71b81-c0de-4ea0-92a5-332e60d78b99-kube-api-access-2ghfw\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.400777 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.400740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fe71b81-c0de-4ea0-92a5-332e60d78b99-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.402107 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.402018    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.403148 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.403120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6ba43672-693f-40b1-831a-545fc97ad5f4-node-exporter-tls\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.403385 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.403364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fe71b81-c0de-4ea0-92a5-332e60d78b99-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.410706 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.410684    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-842tk\" (UniqueName: \"kubernetes.io/projected/6ba43672-693f-40b1-831a-545fc97ad5f4-kube-api-access-842tk\") pod \"node-exporter-vlm28\" (UID: \"6ba43672-693f-40b1-831a-545fc97ad5f4\") " pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.413909 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.413887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ghfw\" (UniqueName: \"kubernetes.io/projected/3fe71b81-c0de-4ea0-92a5-332e60d78b99-kube-api-access-2ghfw\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.559849 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.559794    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vlm28"
Jul 22 18:45:09.570490 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:45:09.570468    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba43672_693f_40b1_831a_545fc97ad5f4.slice/crio-1409c86d5c2be8850165f0d11acc3e8edddb13f7176652911fbc624d9f114674 WatchSource:0}: Error finding container 1409c86d5c2be8850165f0d11acc3e8edddb13f7176652911fbc624d9f114674: Status 404 returned error can't find the container with id 1409c86d5c2be8850165f0d11acc3e8edddb13f7176652911fbc624d9f114674
Jul 22 18:45:09.649449 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.649423    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vlm28" event={"ID":"6ba43672-693f-40b1-831a-545fc97ad5f4","Type":"ContainerStarted","Data":"1409c86d5c2be8850165f0d11acc3e8edddb13f7176652911fbc624d9f114674"}
Jul 22 18:45:09.904835 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.904809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3fe71b81-c0de-4ea0-92a5-332e60d78b99-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:09.907070 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:09.907051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3fe71b81-c0de-4ea0-92a5-332e60d78b99-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-htcvp\" (UID: \"3fe71b81-c0de-4ea0-92a5-332e60d78b99\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:10.110141 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:10.110108    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"
Jul 22 18:45:10.257338 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:10.257308    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp"]
Jul 22 18:45:10.374118 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:45:10.374083    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe71b81_c0de_4ea0_92a5_332e60d78b99.slice/crio-4e46c24a5a3e63ee651fd1898c1367226c69d794144356d8abb5f7baa7b1721c WatchSource:0}: Error finding container 4e46c24a5a3e63ee651fd1898c1367226c69d794144356d8abb5f7baa7b1721c: Status 404 returned error can't find the container with id 4e46c24a5a3e63ee651fd1898c1367226c69d794144356d8abb5f7baa7b1721c
Jul 22 18:45:10.589028 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:10.589002    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65d7fc9fc6-j56rc"
Jul 22 18:45:10.654691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:10.654656    2576 generic.go:356] "Generic (PLEG): container finished" podID="6ba43672-693f-40b1-831a-545fc97ad5f4" containerID="188690ee80c67431660d2f4d37ae20c7651c8509e295294d5721b2f89ee23b33" exitCode=0
Jul 22 18:45:10.654864 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:10.654751    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vlm28" event={"ID":"6ba43672-693f-40b1-831a-545fc97ad5f4","Type":"ContainerDied","Data":"188690ee80c67431660d2f4d37ae20c7651c8509e295294d5721b2f89ee23b33"}
Jul 22 18:45:10.657016 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:10.656979    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp" event={"ID":"3fe71b81-c0de-4ea0-92a5-332e60d78b99","Type":"ContainerStarted","Data":"cb52332184f4cbc6460c68d02be4501f8a4ff124feff8a61329e392dd6e0d3a0"}
Jul 22 18:45:10.657119 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:10.657038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp" event={"ID":"3fe71b81-c0de-4ea0-92a5-332e60d78b99","Type":"ContainerStarted","Data":"8d708acc2755e4a2434e028b67e6aec168cc36d0f42ea10b936c1f9818c634b1"}
Jul 22 18:45:10.657119 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:10.657068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp" event={"ID":"3fe71b81-c0de-4ea0-92a5-332e60d78b99","Type":"ContainerStarted","Data":"4e46c24a5a3e63ee651fd1898c1367226c69d794144356d8abb5f7baa7b1721c"}
Jul 22 18:45:11.667778 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:11.667727    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vlm28" event={"ID":"6ba43672-693f-40b1-831a-545fc97ad5f4","Type":"ContainerStarted","Data":"48721df6fa1df1d9c66d93823d0395f5c8dadc7ae756db27839e6458e76c2754"}
Jul 22 18:45:11.668205 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:11.667814    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vlm28" event={"ID":"6ba43672-693f-40b1-831a-545fc97ad5f4","Type":"ContainerStarted","Data":"f9afe28f55b1774c47ce83026f2408da3a12ce2a604c4d51aae9b9117b82450b"}
Jul 22 18:45:11.682545 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:11.682513    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:45:11.691615 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:11.690977    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vlm28" podStartSLOduration=1.772198299 podStartE2EDuration="2.690959567s" totalImagesPullingTime="849.045761ms" totalInitContainerRuntime="69.715507ms" isStatefulPod=true podCreationTimestamp="2026-07-22 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:11.690127012 +0000 UTC m=+312.580667946" watchObservedRunningTime="2026-07-22 18:45:11.690959567 +0000 UTC m=+312.581500500"
Jul 22 18:45:14.681971 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:14.681926    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cg27c" event={"ID":"70d227ba-ecb7-446b-a997-e770e52d894d","Type":"ContainerStarted","Data":"3f84dfa38ad31acfaa7215dfc919a16adee98d27c3b22ea97b908b08dcaec440"}
Jul 22 18:45:14.683851 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:14.683829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp" event={"ID":"3fe71b81-c0de-4ea0-92a5-332e60d78b99","Type":"ContainerStarted","Data":"9c8267cff5216497b13bd30f16a29532d4cb4b20f6c4b9efe22880be512391f1"}
Jul 22 18:45:14.700189 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:14.700148    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cg27c" podStartSLOduration=33.589895341 podStartE2EDuration="49.700136153s" totalImagesPullingTime="16.110240812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:44:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:14.699332718 +0000 UTC m=+315.589873649" watchObservedRunningTime="2026-07-22 18:45:14.700136153 +0000 UTC m=+315.590677084"
Jul 22 18:45:14.716970 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:14.716931    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-htcvp" podStartSLOduration=2.486571122 podStartE2EDuration="5.716918219s" totalImagesPullingTime="3.230347097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:14.716276992 +0000 UTC m=+315.606817923" watchObservedRunningTime="2026-07-22 18:45:14.716918219 +0000 UTC m=+315.607459150"
Jul 22 18:45:18.069765 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:18.069724    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" podUID="d92e7e20-9642-473e-9903-b29b4bcfc456" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 22 18:45:18.070127 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:18.069796    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j"
Jul 22 18:45:18.070340 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:18.070308    2576 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"ca1244ca0a9cbc17bf93e24d9f49c52b328cfc0c283b0d52e8034a0ef4cddc5f"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 22 18:45:18.070377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:18.070362    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" podUID="d92e7e20-9642-473e-9903-b29b4bcfc456" containerName="service-proxy" containerID="cri-o://ca1244ca0a9cbc17bf93e24d9f49c52b328cfc0c283b0d52e8034a0ef4cddc5f" gracePeriod=30
Jul 22 18:45:18.697981 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:18.697951    2576 generic.go:356] "Generic (PLEG): container finished" podID="d92e7e20-9642-473e-9903-b29b4bcfc456" containerID="ca1244ca0a9cbc17bf93e24d9f49c52b328cfc0c283b0d52e8034a0ef4cddc5f" exitCode=2
Jul 22 18:45:18.698122 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:18.698027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" event={"ID":"d92e7e20-9642-473e-9903-b29b4bcfc456","Type":"ContainerDied","Data":"ca1244ca0a9cbc17bf93e24d9f49c52b328cfc0c283b0d52e8034a0ef4cddc5f"}
Jul 22 18:45:18.698122 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:18.698069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b889d84cd-s8m5j" event={"ID":"d92e7e20-9642-473e-9903-b29b4bcfc456","Type":"ContainerStarted","Data":"2bece6790df9115e9f13e402f96c5b3acb3702dea9120254396c9c9a998e43c9"}
Jul 22 18:45:45.981110 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:45:45.981078    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9h9qz_5241d666-4a13-4c9d-8ff2-a7448dce1332/serve-healthcheck-canary/0.log"
Jul 22 18:46:13.656735 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:46:13.656644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:46:13.658935 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:46:13.658914    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881e051a-b4aa-4d47-b389-0f3dd300de2f-metrics-certs\") pod \"network-metrics-daemon-xd9sx\" (UID: \"881e051a-b4aa-4d47-b389-0f3dd300de2f\") " pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:46:13.787691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:46:13.787662    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2vpr\""
Jul 22 18:46:13.793884 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:46:13.793865    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xd9sx"
Jul 22 18:46:13.922985 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:46:13.922960    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xd9sx"]
Jul 22 18:46:13.924621 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:46:13.924593    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881e051a_b4aa_4d47_b389_0f3dd300de2f.slice/crio-ceafe096927ca86da85ea2c15f1959b242aaacdb0d60497a28d0e28de5b86b95 WatchSource:0}: Error finding container ceafe096927ca86da85ea2c15f1959b242aaacdb0d60497a28d0e28de5b86b95: Status 404 returned error can't find the container with id ceafe096927ca86da85ea2c15f1959b242aaacdb0d60497a28d0e28de5b86b95
Jul 22 18:46:14.867354 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:46:14.867316    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xd9sx" event={"ID":"881e051a-b4aa-4d47-b389-0f3dd300de2f","Type":"ContainerStarted","Data":"ceafe096927ca86da85ea2c15f1959b242aaacdb0d60497a28d0e28de5b86b95"}
Jul 22 18:46:16.876135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:46:16.876100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xd9sx" event={"ID":"881e051a-b4aa-4d47-b389-0f3dd300de2f","Type":"ContainerStarted","Data":"3895e290d378de9e4b461ddce2de729b016cfcce29efc2ef70f625310a41da26"}
Jul 22 18:46:16.876716 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:46:16.876152    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xd9sx" event={"ID":"881e051a-b4aa-4d47-b389-0f3dd300de2f","Type":"ContainerStarted","Data":"994f39bab3300c5c32c927f4ea710475d28ee641a8abd578979d96862e3a50da"}
Jul 22 18:46:16.901536 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:46:16.901487    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xd9sx" podStartSLOduration=374.931586389 podStartE2EDuration="6m16.901473768s" totalImagesPullingTime="1.969887379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:46:16.899886609 +0000 UTC m=+377.790427554" watchObservedRunningTime="2026-07-22 18:46:16.901473768 +0000 UTC m=+377.792014699"
Jul 22 18:48:52.396655 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.396540    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"]
Jul 22 18:48:52.398682 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.398659    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"
Jul 22 18:48:52.401417 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.401400    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2nn6h\""
Jul 22 18:48:52.401618 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.401603    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 18:48:52.408029 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.408005    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 18:48:52.411243 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.411221    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"]
Jul 22 18:48:52.504233 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.504208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8lr6\" (UniqueName: \"kubernetes.io/projected/43ec3b4f-8632-4955-ba17-3765d176bc16-kube-api-access-f8lr6\") pod \"openshift-lws-operator-fd8ccff4c-fhsx8\" (UID: \"43ec3b4f-8632-4955-ba17-3765d176bc16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"
Jul 22 18:48:52.504356 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.504244    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43ec3b4f-8632-4955-ba17-3765d176bc16-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fhsx8\" (UID: \"43ec3b4f-8632-4955-ba17-3765d176bc16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"
Jul 22 18:48:52.604986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.604958    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8lr6\" (UniqueName: \"kubernetes.io/projected/43ec3b4f-8632-4955-ba17-3765d176bc16-kube-api-access-f8lr6\") pod \"openshift-lws-operator-fd8ccff4c-fhsx8\" (UID: \"43ec3b4f-8632-4955-ba17-3765d176bc16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"
Jul 22 18:48:52.605090 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.604992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43ec3b4f-8632-4955-ba17-3765d176bc16-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fhsx8\" (UID: \"43ec3b4f-8632-4955-ba17-3765d176bc16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"
Jul 22 18:48:52.605360 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.605344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43ec3b4f-8632-4955-ba17-3765d176bc16-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fhsx8\" (UID: \"43ec3b4f-8632-4955-ba17-3765d176bc16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"
Jul 22 18:48:52.614155 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.614135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8lr6\" (UniqueName: \"kubernetes.io/projected/43ec3b4f-8632-4955-ba17-3765d176bc16-kube-api-access-f8lr6\") pod \"openshift-lws-operator-fd8ccff4c-fhsx8\" (UID: \"43ec3b4f-8632-4955-ba17-3765d176bc16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"
Jul 22 18:48:52.710794 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.710730    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"
Jul 22 18:48:52.830249 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:52.830230    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8"]
Jul 22 18:48:52.832761 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:48:52.832733    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43ec3b4f_8632_4955_ba17_3765d176bc16.slice/crio-2df7d48dacc7fd5315b3a2d0840e5fafbf6f276e7e24083e4ff1ab4b4a35afa8 WatchSource:0}: Error finding container 2df7d48dacc7fd5315b3a2d0840e5fafbf6f276e7e24083e4ff1ab4b4a35afa8: Status 404 returned error can't find the container with id 2df7d48dacc7fd5315b3a2d0840e5fafbf6f276e7e24083e4ff1ab4b4a35afa8
Jul 22 18:48:53.384292 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:53.384261    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8" event={"ID":"43ec3b4f-8632-4955-ba17-3765d176bc16","Type":"ContainerStarted","Data":"2df7d48dacc7fd5315b3a2d0840e5fafbf6f276e7e24083e4ff1ab4b4a35afa8"}
Jul 22 18:48:58.404487 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:58.404453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8" event={"ID":"43ec3b4f-8632-4955-ba17-3765d176bc16","Type":"ContainerStarted","Data":"2404675c9c171bf31f3807a993d91c7b79fa75253667ba6b32c79fb28d36214b"}
Jul 22 18:48:58.421495 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:48:58.421447    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhsx8" podStartSLOduration=1.3257647129999999 podStartE2EDuration="6.421431456s" totalImagesPullingTime="5.095666743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:48:58.421142375 +0000 UTC m=+539.311683309" watchObservedRunningTime="2026-07-22 18:48:58.421431456 +0000 UTC m=+539.311972388"
Jul 22 18:49:17.980455 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:17.980415    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"]
Jul 22 18:49:17.983823 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:17.983802    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:17.987671 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:17.987647    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 18:49:17.987793 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:17.987672    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 18:49:17.987793 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:17.987675    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bkd9t\""
Jul 22 18:49:17.987793 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:17.987762    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 18:49:17.987985 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:17.987969    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 18:49:18.010098 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.010072    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"]
Jul 22 18:49:18.084463 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.084432    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6126ec5f-9406-43de-b0d5-8b37a22000c8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.084463 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.084468    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6126ec5f-9406-43de-b0d5-8b37a22000c8-tmp\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.084679 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.084510    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6126ec5f-9406-43de-b0d5-8b37a22000c8-manifests\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.084679 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.084574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6126ec5f-9406-43de-b0d5-8b37a22000c8-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.084679 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.084600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmvtd\" (UniqueName: \"kubernetes.io/projected/6126ec5f-9406-43de-b0d5-8b37a22000c8-kube-api-access-jmvtd\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.185387 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.185355    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6126ec5f-9406-43de-b0d5-8b37a22000c8-manifests\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.185573 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.185395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6126ec5f-9406-43de-b0d5-8b37a22000c8-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.185573 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.185413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmvtd\" (UniqueName: \"kubernetes.io/projected/6126ec5f-9406-43de-b0d5-8b37a22000c8-kube-api-access-jmvtd\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.185573 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.185446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6126ec5f-9406-43de-b0d5-8b37a22000c8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.185573 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.185496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6126ec5f-9406-43de-b0d5-8b37a22000c8-tmp\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.185832 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.185807    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6126ec5f-9406-43de-b0d5-8b37a22000c8-manifests\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.187839 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.187813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6126ec5f-9406-43de-b0d5-8b37a22000c8-tmp\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.188009 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.187990    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6126ec5f-9406-43de-b0d5-8b37a22000c8-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.188072 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.188010    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6126ec5f-9406-43de-b0d5-8b37a22000c8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.193943 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.193920    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmvtd\" (UniqueName: \"kubernetes.io/projected/6126ec5f-9406-43de-b0d5-8b37a22000c8-kube-api-access-jmvtd\") pod \"opendatahub-operator-controller-manager-f9bb58876-qwvll\" (UID: \"6126ec5f-9406-43de-b0d5-8b37a22000c8\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.293914 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.293827    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:18.422747 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.422724    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"]
Jul 22 18:49:18.425651 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:49:18.425606    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6126ec5f_9406_43de_b0d5_8b37a22000c8.slice/crio-d43da957333a0fac8d7d183383a5c2017a8960165ef56d4813d739ea7521bd69 WatchSource:0}: Error finding container d43da957333a0fac8d7d183383a5c2017a8960165ef56d4813d739ea7521bd69: Status 404 returned error can't find the container with id d43da957333a0fac8d7d183383a5c2017a8960165ef56d4813d739ea7521bd69
Jul 22 18:49:18.468130 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:18.468100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll" event={"ID":"6126ec5f-9406-43de-b0d5-8b37a22000c8","Type":"ContainerStarted","Data":"d43da957333a0fac8d7d183383a5c2017a8960165ef56d4813d739ea7521bd69"}
Jul 22 18:49:22.485173 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:22.485135    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll" event={"ID":"6126ec5f-9406-43de-b0d5-8b37a22000c8","Type":"ContainerStarted","Data":"f78ec4c3edf7c2e387717f9ac8faf17c1bb14a8d39adb1ffb58a8b73acb15839"}
Jul 22 18:49:23.489656 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:23.489600    2576 generic.go:356] "Generic (PLEG): container finished" podID="6126ec5f-9406-43de-b0d5-8b37a22000c8" containerID="f78ec4c3edf7c2e387717f9ac8faf17c1bb14a8d39adb1ffb58a8b73acb15839" exitCode=0
Jul 22 18:49:23.490068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:23.489670    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll" event={"ID":"6126ec5f-9406-43de-b0d5-8b37a22000c8","Type":"ContainerDied","Data":"f78ec4c3edf7c2e387717f9ac8faf17c1bb14a8d39adb1ffb58a8b73acb15839"}
Jul 22 18:49:24.494423 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:24.494392    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll" event={"ID":"6126ec5f-9406-43de-b0d5-8b37a22000c8","Type":"ContainerStarted","Data":"03b2a8464c5f0d31b5185837ab7945e1cffe5effdd40a059dabec9c0f7273fe5"}
Jul 22 18:49:24.494813 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:24.494543    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:24.517592 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:24.517545    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll" podStartSLOduration=2.833705597 podStartE2EDuration="7.517530344s" totalImagesPullingTime="3.676214457s" totalInitContainerRuntime="1.00761029s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:24.515968643 +0000 UTC m=+565.406509575" watchObservedRunningTime="2026-07-22 18:49:24.517530344 +0000 UTC m=+565.408071275"
Jul 22 18:49:35.499636 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:35.499604    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-qwvll"
Jul 22 18:49:43.294205 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.294168    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"]
Jul 22 18:49:43.302102 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.302061    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.304697 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.304673    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 18:49:43.304854 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.304735    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 18:49:43.304854 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.304815    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 18:49:43.305206 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.305189    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 18:49:43.310160 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.310092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"]
Jul 22 18:49:43.363879 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.363836    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-tls-certs\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.364028 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.363909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-tmp\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.364028 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.363949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s456q\" (UniqueName: \"kubernetes.io/projected/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-kube-api-access-s456q\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.464428 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.464396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-tmp\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.464578 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.464448    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s456q\" (UniqueName: \"kubernetes.io/projected/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-kube-api-access-s456q\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.464578 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.464479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-tls-certs\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.464691 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:49:43.464615    2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 22 18:49:43.464737 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:49:43.464708    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-tls-certs podName:e64c1ba6-a192-4278-bbfd-7cdfa9c64af4 nodeName:}" failed. No retries permitted until 2026-07-22 18:49:43.964686741 +0000 UTC m=+584.855227656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-tls-certs") pod "kube-auth-proxy-779ddf9494-p2scd" (UID: "e64c1ba6-a192-4278-bbfd-7cdfa9c64af4") : secret "kube-auth-proxy-tls" not found
Jul 22 18:49:43.466665 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.466645    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-tmp\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.487560 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.487535    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8xrd8"]
Jul 22 18:49:43.489220 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.489195    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s456q\" (UniqueName: \"kubernetes.io/projected/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-kube-api-access-s456q\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.490922 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.490908    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:43.496113 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.496091    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jbzww\""
Jul 22 18:49:43.496204 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.496183    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 22 18:49:43.506245 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.506225    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8xrd8"]
Jul 22 18:49:43.565269 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.565182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3eb13c22-c477-4dab-99c5-37f8936c122c-cert\") pod \"odh-model-controller-7998c67dc4-8xrd8\" (UID: \"3eb13c22-c477-4dab-99c5-37f8936c122c\") " pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:43.565269 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.565240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk6bb\" (UniqueName: \"kubernetes.io/projected/3eb13c22-c477-4dab-99c5-37f8936c122c-kube-api-access-pk6bb\") pod \"odh-model-controller-7998c67dc4-8xrd8\" (UID: \"3eb13c22-c477-4dab-99c5-37f8936c122c\") " pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:43.666306 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.666275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk6bb\" (UniqueName: \"kubernetes.io/projected/3eb13c22-c477-4dab-99c5-37f8936c122c-kube-api-access-pk6bb\") pod \"odh-model-controller-7998c67dc4-8xrd8\" (UID: \"3eb13c22-c477-4dab-99c5-37f8936c122c\") " pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:43.666503 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.666373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3eb13c22-c477-4dab-99c5-37f8936c122c-cert\") pod \"odh-model-controller-7998c67dc4-8xrd8\" (UID: \"3eb13c22-c477-4dab-99c5-37f8936c122c\") " pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:43.666503 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:49:43.666485    2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 22 18:49:43.666660 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:49:43.666558    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3eb13c22-c477-4dab-99c5-37f8936c122c-cert podName:3eb13c22-c477-4dab-99c5-37f8936c122c nodeName:}" failed. No retries permitted until 2026-07-22 18:49:44.166537964 +0000 UTC m=+585.057078880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3eb13c22-c477-4dab-99c5-37f8936c122c-cert") pod "odh-model-controller-7998c67dc4-8xrd8" (UID: "3eb13c22-c477-4dab-99c5-37f8936c122c") : secret "odh-model-controller-webhook-cert" not found
Jul 22 18:49:43.675510 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.675486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk6bb\" (UniqueName: \"kubernetes.io/projected/3eb13c22-c477-4dab-99c5-37f8936c122c-kube-api-access-pk6bb\") pod \"odh-model-controller-7998c67dc4-8xrd8\" (UID: \"3eb13c22-c477-4dab-99c5-37f8936c122c\") " pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:43.968281 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.968238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-tls-certs\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:43.970907 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:43.970876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e64c1ba6-a192-4278-bbfd-7cdfa9c64af4-tls-certs\") pod \"kube-auth-proxy-779ddf9494-p2scd\" (UID: \"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4\") " pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:44.169216 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.169183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3eb13c22-c477-4dab-99c5-37f8936c122c-cert\") pod \"odh-model-controller-7998c67dc4-8xrd8\" (UID: \"3eb13c22-c477-4dab-99c5-37f8936c122c\") " pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:44.172778 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.172741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3eb13c22-c477-4dab-99c5-37f8936c122c-cert\") pod \"odh-model-controller-7998c67dc4-8xrd8\" (UID: \"3eb13c22-c477-4dab-99c5-37f8936c122c\") " pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:44.213155 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.213125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"
Jul 22 18:49:44.356324 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.356224    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779ddf9494-p2scd"]
Jul 22 18:49:44.359789 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:49:44.359758    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode64c1ba6_a192_4278_bbfd_7cdfa9c64af4.slice/crio-9cb0e31b9c9e05a559a6b79982a6f34e15b180cd5f1ecafe3ba889e0372c8866 WatchSource:0}: Error finding container 9cb0e31b9c9e05a559a6b79982a6f34e15b180cd5f1ecafe3ba889e0372c8866: Status 404 returned error can't find the container with id 9cb0e31b9c9e05a559a6b79982a6f34e15b180cd5f1ecafe3ba889e0372c8866
Jul 22 18:49:44.362897 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.362874    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-78rqr"]
Jul 22 18:49:44.367214 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.367196    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:44.376588 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.376561    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 22 18:49:44.376726 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.376654    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-86b6n\""
Jul 22 18:49:44.383079 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.383058    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-78rqr"]
Jul 22 18:49:44.403043 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.403023    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:44.476701 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.472147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb0aaeba-b112-47ef-a6ac-3fecfc1277c8-tls-certs\") pod \"model-serving-api-54d89bd5fc-78rqr\" (UID: \"fb0aaeba-b112-47ef-a6ac-3fecfc1277c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:44.476701 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.472192    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghbhq\" (UniqueName: \"kubernetes.io/projected/fb0aaeba-b112-47ef-a6ac-3fecfc1277c8-kube-api-access-ghbhq\") pod \"model-serving-api-54d89bd5fc-78rqr\" (UID: \"fb0aaeba-b112-47ef-a6ac-3fecfc1277c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:44.505508 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.505478    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"]
Jul 22 18:49:44.509045 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.509024    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.511620 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.511591    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 18:49:44.511748 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.511692    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 22 18:49:44.511748 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.511735    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 22 18:49:44.511868 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.511700    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 22 18:49:44.511868 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.511699    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cx66g\""
Jul 22 18:49:44.523834 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.523810    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"]
Jul 22 18:49:44.561580 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.561517    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8xrd8"]
Jul 22 18:49:44.563539 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:49:44.563510    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb13c22_c477_4dab_99c5_37f8936c122c.slice/crio-e9a66ab48ffb2f68b01a77f4a778fee17fcd1c6389094f36d3361f35ea3c6f5d WatchSource:0}: Error finding container e9a66ab48ffb2f68b01a77f4a778fee17fcd1c6389094f36d3361f35ea3c6f5d: Status 404 returned error can't find the container with id e9a66ab48ffb2f68b01a77f4a778fee17fcd1c6389094f36d3361f35ea3c6f5d
Jul 22 18:49:44.566005 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.565976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd" event={"ID":"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4","Type":"ContainerStarted","Data":"9cb0e31b9c9e05a559a6b79982a6f34e15b180cd5f1ecafe3ba889e0372c8866"}
Jul 22 18:49:44.573461 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.573435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.573542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.573490    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-cacerts\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.573542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.573530    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76v6z\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-kube-api-access-76v6z\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.573616 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.573570    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.573678 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.573655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb0aaeba-b112-47ef-a6ac-3fecfc1277c8-tls-certs\") pod \"model-serving-api-54d89bd5fc-78rqr\" (UID: \"fb0aaeba-b112-47ef-a6ac-3fecfc1277c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:44.573710 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.573682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbhq\" (UniqueName: \"kubernetes.io/projected/fb0aaeba-b112-47ef-a6ac-3fecfc1277c8-kube-api-access-ghbhq\") pod \"model-serving-api-54d89bd5fc-78rqr\" (UID: \"fb0aaeba-b112-47ef-a6ac-3fecfc1277c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:44.573741 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.573707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/06891eaf-37b0-4288-9abd-a35b6dcbe89f-local-certs\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.573741 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.573724    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.573811 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.573750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-token\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.573811 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:49:44.573801    2576 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 22 18:49:44.573893 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:49:44.573882    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb0aaeba-b112-47ef-a6ac-3fecfc1277c8-tls-certs podName:fb0aaeba-b112-47ef-a6ac-3fecfc1277c8 nodeName:}" failed. No retries permitted until 2026-07-22 18:49:45.073860559 +0000 UTC m=+585.964401474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fb0aaeba-b112-47ef-a6ac-3fecfc1277c8-tls-certs") pod "model-serving-api-54d89bd5fc-78rqr" (UID: "fb0aaeba-b112-47ef-a6ac-3fecfc1277c8") : secret "model-serving-api-tls" not found
Jul 22 18:49:44.589997 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.589969    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbhq\" (UniqueName: \"kubernetes.io/projected/fb0aaeba-b112-47ef-a6ac-3fecfc1277c8-kube-api-access-ghbhq\") pod \"model-serving-api-54d89bd5fc-78rqr\" (UID: \"fb0aaeba-b112-47ef-a6ac-3fecfc1277c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:44.674679 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.674641    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.674889 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.674709    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/06891eaf-37b0-4288-9abd-a35b6dcbe89f-local-certs\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.674889 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.674732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.674889 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.674751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-token\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.674889 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.674774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.674889 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.674816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-cacerts\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.674889 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.674853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76v6z\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-kube-api-access-76v6z\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.675389 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.675364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.677446 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.677413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.677446 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.677421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/06891eaf-37b0-4288-9abd-a35b6dcbe89f-local-certs\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.677767 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.677747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-cacerts\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.677806 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.677790    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.687000 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.686975    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-token\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.687467 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.687444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76v6z\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-kube-api-access-76v6z\") pod \"istiod-openshift-gateway-7574459656-4v45j\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.820271 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.820198    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:44.956594 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:44.956545    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"]
Jul 22 18:49:44.960267 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:49:44.960239    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06891eaf_37b0_4288_9abd_a35b6dcbe89f.slice/crio-5738e1f7098f8ad000f2f5c1c778f6fe5cf5663555171b6a088a4ccadb724268 WatchSource:0}: Error finding container 5738e1f7098f8ad000f2f5c1c778f6fe5cf5663555171b6a088a4ccadb724268: Status 404 returned error can't find the container with id 5738e1f7098f8ad000f2f5c1c778f6fe5cf5663555171b6a088a4ccadb724268
Jul 22 18:49:45.078504 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:45.078400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb0aaeba-b112-47ef-a6ac-3fecfc1277c8-tls-certs\") pod \"model-serving-api-54d89bd5fc-78rqr\" (UID: \"fb0aaeba-b112-47ef-a6ac-3fecfc1277c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:45.080817 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:45.080791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb0aaeba-b112-47ef-a6ac-3fecfc1277c8-tls-certs\") pod \"model-serving-api-54d89bd5fc-78rqr\" (UID: \"fb0aaeba-b112-47ef-a6ac-3fecfc1277c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:45.278097 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:45.278058    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:45.423299 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:45.423211    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-78rqr"]
Jul 22 18:49:45.426653 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:49:45.426587    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0aaeba_b112_47ef_a6ac_3fecfc1277c8.slice/crio-c88fdf52605078e0342c47660ef9e07f36d184a19bb551b83c2f5f22579e7712 WatchSource:0}: Error finding container c88fdf52605078e0342c47660ef9e07f36d184a19bb551b83c2f5f22579e7712: Status 404 returned error can't find the container with id c88fdf52605078e0342c47660ef9e07f36d184a19bb551b83c2f5f22579e7712
Jul 22 18:49:45.572725 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:45.572689    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j" event={"ID":"06891eaf-37b0-4288-9abd-a35b6dcbe89f","Type":"ContainerStarted","Data":"5738e1f7098f8ad000f2f5c1c778f6fe5cf5663555171b6a088a4ccadb724268"}
Jul 22 18:49:45.574302 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:45.574271    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-78rqr" event={"ID":"fb0aaeba-b112-47ef-a6ac-3fecfc1277c8","Type":"ContainerStarted","Data":"c88fdf52605078e0342c47660ef9e07f36d184a19bb551b83c2f5f22579e7712"}
Jul 22 18:49:45.576605 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:45.576094    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8" event={"ID":"3eb13c22-c477-4dab-99c5-37f8936c122c","Type":"ContainerStarted","Data":"e9a66ab48ffb2f68b01a77f4a778fee17fcd1c6389094f36d3361f35ea3c6f5d"}
Jul 22 18:49:49.953069 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:49.953034    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:49:49.953477 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:49.953106    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:49:50.604778 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.604741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd" event={"ID":"e64c1ba6-a192-4278-bbfd-7cdfa9c64af4","Type":"ContainerStarted","Data":"bc8c74ab12677ee28faa49601e85d728e1f0b7524a2370188f48272afe7abeba"}
Jul 22 18:49:50.606348 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.606319    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j" event={"ID":"06891eaf-37b0-4288-9abd-a35b6dcbe89f","Type":"ContainerStarted","Data":"7c38cbb131bd0a61931c4b3265adab550ce5d3f50ac6c0039049fd3a83b54a72"}
Jul 22 18:49:50.606489 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.606375    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:50.607644 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.607596    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-78rqr" event={"ID":"fb0aaeba-b112-47ef-a6ac-3fecfc1277c8","Type":"ContainerStarted","Data":"c6799075099d756d01ebdac41dd3bf3ccbfddc31c77bbec5171a7ebc16bb9d7c"}
Jul 22 18:49:50.607774 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.607704    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:49:50.608990 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.608970    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8" event={"ID":"3eb13c22-c477-4dab-99c5-37f8936c122c","Type":"ContainerStarted","Data":"a109ab564f1fefc528e2f3e4e99046063e19e991a3ad194d771291d471050b15"}
Jul 22 18:49:50.609110 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.609025    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:49:50.626441 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.626393    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-779ddf9494-p2scd" podStartSLOduration=2.028094927 podStartE2EDuration="7.626375913s" totalImagesPullingTime="5.598280986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:50.625701154 +0000 UTC m=+591.516242086" watchObservedRunningTime="2026-07-22 18:49:50.626375913 +0000 UTC m=+591.516916848"
Jul 22 18:49:50.648734 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.648657    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-78rqr" podStartSLOduration=2.128236898 podStartE2EDuration="6.648621911s" totalImagesPullingTime="4.520385013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:50.647434088 +0000 UTC m=+591.537975022" watchObservedRunningTime="2026-07-22 18:49:50.648621911 +0000 UTC m=+591.539162845"
Jul 22 18:49:50.682235 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.682178    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j" podStartSLOduration=1.691813909 podStartE2EDuration="6.682163571s" totalImagesPullingTime="4.990349662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:50.680952481 +0000 UTC m=+591.571493422" watchObservedRunningTime="2026-07-22 18:49:50.682163571 +0000 UTC m=+591.572704501"
Jul 22 18:49:50.716895 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:50.716837    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8" podStartSLOduration=2.332911045 podStartE2EDuration="7.71682239s" totalImagesPullingTime="5.383911345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:50.713581633 +0000 UTC m=+591.604122566" watchObservedRunningTime="2026-07-22 18:49:50.71682239 +0000 UTC m=+591.607363321"
Jul 22 18:49:51.517733 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.517692    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"]
Jul 22 18:49:51.521108 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.521087    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.526292 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.526270    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ffl8l\""
Jul 22 18:49:51.536293 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.536269    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"]
Jul 22 18:49:51.614585 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.614561    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:49:51.633569 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.633538    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.633704 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.633595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.633704 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.633682    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.633789 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.633730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.634336 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.633877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.634454 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.634363    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.634454 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.634405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fvkp\" (UniqueName: \"kubernetes.io/projected/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-kube-api-access-2fvkp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.634570 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.634543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.635004 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.634983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735390 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735390 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735567 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735411    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735567 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735567 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735551    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735754 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735754 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735754 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735897 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735759    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fvkp\" (UniqueName: \"kubernetes.io/projected/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-kube-api-access-2fvkp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735897 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.735897 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.736047 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.735895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.736158 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.736142    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.736262 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.736245    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.737937 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.737912    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.738124 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.738102    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.743942 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.743920    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fvkp\" (UniqueName: \"kubernetes.io/projected/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-kube-api-access-2fvkp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.744068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.744047    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79c6f2b5-cb56-4b6d-8b3f-be315b55ef73-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c6gs\" (UID: \"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.831201 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.831132    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:51.956768 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:51.956740    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"]
Jul 22 18:49:51.958971 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:49:51.958946    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c6f2b5_cb56_4b6d_8b3f_be315b55ef73.slice/crio-d8f3a88f3b5cc75d0b93db62395b1e07ed45a1a733fc02b8a4dfe2942397ca6c WatchSource:0}: Error finding container d8f3a88f3b5cc75d0b93db62395b1e07ed45a1a733fc02b8a4dfe2942397ca6c: Status 404 returned error can't find the container with id d8f3a88f3b5cc75d0b93db62395b1e07ed45a1a733fc02b8a4dfe2942397ca6c
Jul 22 18:49:52.620938 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:52.620900    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs" event={"ID":"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73","Type":"ContainerStarted","Data":"d8f3a88f3b5cc75d0b93db62395b1e07ed45a1a733fc02b8a4dfe2942397ca6c"}
Jul 22 18:49:54.350521 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.350469    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:49:54.350820 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.350564    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:49:54.350820 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.350590    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:49:54.631210 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.631176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs" event={"ID":"79c6f2b5-cb56-4b6d-8b3f-be315b55ef73","Type":"ContainerStarted","Data":"a25d571e648243a4eea9aab8d9b13200b0f727ba9e9c192bb58e782b69f197b4"}
Jul 22 18:49:54.656411 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.656364    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs" podStartSLOduration=1.267486198 podStartE2EDuration="3.656349531s" totalImagesPullingTime="2.388863333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:54.655621022 +0000 UTC m=+595.546161955" watchObservedRunningTime="2026-07-22 18:49:54.656349531 +0000 UTC m=+595.546890492"
Jul 22 18:49:54.832244 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.832208    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:54.973242 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.973165    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"]
Jul 22 18:49:54.981295 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.981270    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:49:54.986104 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.986072    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 22 18:49:54.986237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.986221    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-hwj8l\""
Jul 22 18:49:54.995775 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:54.995751    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"]
Jul 22 18:49:55.064817 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.064780    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfkgd\" (UniqueName: \"kubernetes.io/projected/8a49525f-d272-43bf-adec-c67966c62812-kube-api-access-tfkgd\") pod \"llmisvc-controller-manager-777c6bd7d5-bmwbj\" (UID: \"8a49525f-d272-43bf-adec-c67966c62812\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:49:55.064965 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.064891    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a49525f-d272-43bf-adec-c67966c62812-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bmwbj\" (UID: \"8a49525f-d272-43bf-adec-c67966c62812\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:49:55.166235 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.166204    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a49525f-d272-43bf-adec-c67966c62812-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bmwbj\" (UID: \"8a49525f-d272-43bf-adec-c67966c62812\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:49:55.166371 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.166269    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfkgd\" (UniqueName: \"kubernetes.io/projected/8a49525f-d272-43bf-adec-c67966c62812-kube-api-access-tfkgd\") pod \"llmisvc-controller-manager-777c6bd7d5-bmwbj\" (UID: \"8a49525f-d272-43bf-adec-c67966c62812\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:49:55.168584 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.168547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a49525f-d272-43bf-adec-c67966c62812-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bmwbj\" (UID: \"8a49525f-d272-43bf-adec-c67966c62812\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:49:55.176067 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.176040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfkgd\" (UniqueName: \"kubernetes.io/projected/8a49525f-d272-43bf-adec-c67966c62812-kube-api-access-tfkgd\") pod \"llmisvc-controller-manager-777c6bd7d5-bmwbj\" (UID: \"8a49525f-d272-43bf-adec-c67966c62812\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:49:55.291323 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.291253    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:49:55.465214 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.465172    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"]
Jul 22 18:49:55.467897 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:49:55.467871    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8a49525f_d272_43bf_adec_c67966c62812.slice/crio-8fe7469d07b55a21f29509730ca142e217a8ab9081c06bd66b5a76346a559070 WatchSource:0}: Error finding container 8fe7469d07b55a21f29509730ca142e217a8ab9081c06bd66b5a76346a559070: Status 404 returned error can't find the container with id 8fe7469d07b55a21f29509730ca142e217a8ab9081c06bd66b5a76346a559070
Jul 22 18:49:55.636776 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.636748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj" event={"ID":"8a49525f-d272-43bf-adec-c67966c62812","Type":"ContainerStarted","Data":"8fe7469d07b55a21f29509730ca142e217a8ab9081c06bd66b5a76346a559070"}
Jul 22 18:49:55.835902 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:55.835872    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:56.644593 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:56.644558    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:56.645648 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:56.645616    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c6gs"
Jul 22 18:49:59.658465 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:59.658424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj" event={"ID":"8a49525f-d272-43bf-adec-c67966c62812","Type":"ContainerStarted","Data":"bc00f63170f6e32a3939c6074b19dc6c0a6f356063a39ee3cb022764a776bcc4"}
Jul 22 18:49:59.658936 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:59.658558    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:49:59.707878 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:49:59.707830    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj" podStartSLOduration=2.517538312 podStartE2EDuration="5.707813379s" totalImagesPullingTime="3.190275067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:59.706450546 +0000 UTC m=+600.596991478" watchObservedRunningTime="2026-07-22 18:49:59.707813379 +0000 UTC m=+600.598354310"
Jul 22 18:50:01.511515 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.511480    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"]
Jul 22 18:50:01.511893 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.511706    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j" podUID="06891eaf-37b0-4288-9abd-a35b6dcbe89f" containerName="discovery" containerID="cri-o://7c38cbb131bd0a61931c4b3265adab550ce5d3f50ac6c0039049fd3a83b54a72" gracePeriod=30
Jul 22 18:50:01.615427 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.615403    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-8xrd8"
Jul 22 18:50:01.618472 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.618451    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-78rqr"
Jul 22 18:50:01.669801 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.669661    2576 generic.go:356] "Generic (PLEG): container finished" podID="06891eaf-37b0-4288-9abd-a35b6dcbe89f" containerID="7c38cbb131bd0a61931c4b3265adab550ce5d3f50ac6c0039049fd3a83b54a72" exitCode=0
Jul 22 18:50:01.669801 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.669741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j" event={"ID":"06891eaf-37b0-4288-9abd-a35b6dcbe89f","Type":"ContainerDied","Data":"7c38cbb131bd0a61931c4b3265adab550ce5d3f50ac6c0039049fd3a83b54a72"}
Jul 22 18:50:01.778194 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.778171    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:50:01.825974 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.825947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-token\") pod \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") "
Jul 22 18:50:01.826087 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.826020    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76v6z\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-kube-api-access-76v6z\") pod \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") "
Jul 22 18:50:01.826087 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.826048    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-kubeconfig\") pod \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") "
Jul 22 18:50:01.826087 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.826064    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-cacerts\") pod \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") "
Jul 22 18:50:01.826286 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.826103    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-ca-configmap\") pod \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") "
Jul 22 18:50:01.826286 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.826136    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-dns-cert\") pod \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") "
Jul 22 18:50:01.826286 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.826171    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/06891eaf-37b0-4288-9abd-a35b6dcbe89f-local-certs\") pod \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\" (UID: \"06891eaf-37b0-4288-9abd-a35b6dcbe89f\") "
Jul 22 18:50:01.826869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.826838    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "06891eaf-37b0-4288-9abd-a35b6dcbe89f" (UID: "06891eaf-37b0-4288-9abd-a35b6dcbe89f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 18:50:01.828396 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.828368    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-cacerts" (OuterVolumeSpecName: "cacerts") pod "06891eaf-37b0-4288-9abd-a35b6dcbe89f" (UID: "06891eaf-37b0-4288-9abd-a35b6dcbe89f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:50:01.828495 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.828463    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-kube-api-access-76v6z" (OuterVolumeSpecName: "kube-api-access-76v6z") pod "06891eaf-37b0-4288-9abd-a35b6dcbe89f" (UID: "06891eaf-37b0-4288-9abd-a35b6dcbe89f"). InnerVolumeSpecName "kube-api-access-76v6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:50:01.828559 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.828540    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-token" (OuterVolumeSpecName: "istio-token") pod "06891eaf-37b0-4288-9abd-a35b6dcbe89f" (UID: "06891eaf-37b0-4288-9abd-a35b6dcbe89f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:50:01.828617 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.828599    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "06891eaf-37b0-4288-9abd-a35b6dcbe89f" (UID: "06891eaf-37b0-4288-9abd-a35b6dcbe89f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:50:01.828694 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.828618    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "06891eaf-37b0-4288-9abd-a35b6dcbe89f" (UID: "06891eaf-37b0-4288-9abd-a35b6dcbe89f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:50:01.828694 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.828651    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06891eaf-37b0-4288-9abd-a35b6dcbe89f-local-certs" (OuterVolumeSpecName: "local-certs") pod "06891eaf-37b0-4288-9abd-a35b6dcbe89f" (UID: "06891eaf-37b0-4288-9abd-a35b6dcbe89f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 18:50:01.926912 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.926877    2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-kubeconfig\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:50:01.926912 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.926906    2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-cacerts\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:50:01.926912 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.926916    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-ca-configmap\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:50:01.927104 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.926927    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-csr-dns-cert\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:50:01.927104 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.926938    2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/06891eaf-37b0-4288-9abd-a35b6dcbe89f-local-certs\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:50:01.927104 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.926947    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-istio-token\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:50:01.927104 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:01.926954    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76v6z\" (UniqueName: \"kubernetes.io/projected/06891eaf-37b0-4288-9abd-a35b6dcbe89f-kube-api-access-76v6z\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:50:02.675193 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:02.675158    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j" event={"ID":"06891eaf-37b0-4288-9abd-a35b6dcbe89f","Type":"ContainerDied","Data":"5738e1f7098f8ad000f2f5c1c778f6fe5cf5663555171b6a088a4ccadb724268"}
Jul 22 18:50:02.675193 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:02.675196    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"
Jul 22 18:50:02.675676 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:02.675229    2576 scope.go:117] "RemoveContainer" containerID="7c38cbb131bd0a61931c4b3265adab550ce5d3f50ac6c0039049fd3a83b54a72"
Jul 22 18:50:02.697059 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:02.697034    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"]
Jul 22 18:50:02.700854 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:02.700834    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4v45j"]
Jul 22 18:50:03.686864 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:03.686836    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06891eaf-37b0-4288-9abd-a35b6dcbe89f" path="/var/lib/kubelet/pods/06891eaf-37b0-4288-9abd-a35b6dcbe89f/volumes"
Jul 22 18:50:30.665159 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:30.665086    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bmwbj"
Jul 22 18:50:39.319457 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.319418    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nrkt"]
Jul 22 18:50:39.319927 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.319838    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06891eaf-37b0-4288-9abd-a35b6dcbe89f" containerName="discovery"
Jul 22 18:50:39.319927 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.319856    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="06891eaf-37b0-4288-9abd-a35b6dcbe89f" containerName="discovery"
Jul 22 18:50:39.320035 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.319934    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="06891eaf-37b0-4288-9abd-a35b6dcbe89f" containerName="discovery"
Jul 22 18:50:39.323007 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.322980    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt"
Jul 22 18:50:39.325499 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.325479    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 18:50:39.326793 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.326769    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 18:50:39.326911 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.326801    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-w6crd\""
Jul 22 18:50:39.332667 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.332644    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nrkt"]
Jul 22 18:50:39.410854 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.410821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbff5\" (UniqueName: \"kubernetes.io/projected/95f18735-294f-44fe-8004-dbbcd6b71402-kube-api-access-kbff5\") pod \"kuadrant-operator-catalog-7nrkt\" (UID: \"95f18735-294f-44fe-8004-dbbcd6b71402\") " pod="kuadrant-system/kuadrant-operator-catalog-7nrkt"
Jul 22 18:50:39.511967 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.511936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbff5\" (UniqueName: \"kubernetes.io/projected/95f18735-294f-44fe-8004-dbbcd6b71402-kube-api-access-kbff5\") pod \"kuadrant-operator-catalog-7nrkt\" (UID: \"95f18735-294f-44fe-8004-dbbcd6b71402\") " pod="kuadrant-system/kuadrant-operator-catalog-7nrkt"
Jul 22 18:50:39.520795 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.520771    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbff5\" (UniqueName: \"kubernetes.io/projected/95f18735-294f-44fe-8004-dbbcd6b71402-kube-api-access-kbff5\") pod \"kuadrant-operator-catalog-7nrkt\" (UID: \"95f18735-294f-44fe-8004-dbbcd6b71402\") " pod="kuadrant-system/kuadrant-operator-catalog-7nrkt"
Jul 22 18:50:39.634121 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.634097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt"
Jul 22 18:50:39.678397 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.678369    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nrkt"]
Jul 22 18:50:39.762133 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.762110    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nrkt"]
Jul 22 18:50:39.764515 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:50:39.764490    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f18735_294f_44fe_8004_dbbcd6b71402.slice/crio-25ff64dab3d65f4c598b472ad53179c28d9bcbc324a922871248c939e253ea5e WatchSource:0}: Error finding container 25ff64dab3d65f4c598b472ad53179c28d9bcbc324a922871248c939e253ea5e: Status 404 returned error can't find the container with id 25ff64dab3d65f4c598b472ad53179c28d9bcbc324a922871248c939e253ea5e
Jul 22 18:50:39.765860 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.765837    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 18:50:39.827158 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.827127    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt" event={"ID":"95f18735-294f-44fe-8004-dbbcd6b71402","Type":"ContainerStarted","Data":"25ff64dab3d65f4c598b472ad53179c28d9bcbc324a922871248c939e253ea5e"}
Jul 22 18:50:39.887606 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.887537    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h4qpd"]
Jul 22 18:50:39.890223 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.890203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h4qpd"
Jul 22 18:50:39.898269 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.898244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h4qpd"]
Jul 22 18:50:39.915810 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:39.915789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj4n7\" (UniqueName: \"kubernetes.io/projected/36fac68c-df61-4246-b37c-66c8772335dc-kube-api-access-lj4n7\") pod \"kuadrant-operator-catalog-h4qpd\" (UID: \"36fac68c-df61-4246-b37c-66c8772335dc\") " pod="kuadrant-system/kuadrant-operator-catalog-h4qpd"
Jul 22 18:50:40.016703 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:40.016679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj4n7\" (UniqueName: \"kubernetes.io/projected/36fac68c-df61-4246-b37c-66c8772335dc-kube-api-access-lj4n7\") pod \"kuadrant-operator-catalog-h4qpd\" (UID: \"36fac68c-df61-4246-b37c-66c8772335dc\") " pod="kuadrant-system/kuadrant-operator-catalog-h4qpd"
Jul 22 18:50:40.025215 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:40.025195    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj4n7\" (UniqueName: \"kubernetes.io/projected/36fac68c-df61-4246-b37c-66c8772335dc-kube-api-access-lj4n7\") pod \"kuadrant-operator-catalog-h4qpd\" (UID: \"36fac68c-df61-4246-b37c-66c8772335dc\") " pod="kuadrant-system/kuadrant-operator-catalog-h4qpd"
Jul 22 18:50:40.199170 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:40.199100    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h4qpd"
Jul 22 18:50:40.324542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:40.324513    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h4qpd"]
Jul 22 18:50:40.326810 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:50:40.326778    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36fac68c_df61_4246_b37c_66c8772335dc.slice/crio-1001b88404718e31bede2454172c7bea8e06742e9d4ad41846839df2ab3bff34 WatchSource:0}: Error finding container 1001b88404718e31bede2454172c7bea8e06742e9d4ad41846839df2ab3bff34: Status 404 returned error can't find the container with id 1001b88404718e31bede2454172c7bea8e06742e9d4ad41846839df2ab3bff34
Jul 22 18:50:40.836661 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:40.836562    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h4qpd" event={"ID":"36fac68c-df61-4246-b37c-66c8772335dc","Type":"ContainerStarted","Data":"1001b88404718e31bede2454172c7bea8e06742e9d4ad41846839df2ab3bff34"}
Jul 22 18:50:42.846360 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:42.846318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt" event={"ID":"95f18735-294f-44fe-8004-dbbcd6b71402","Type":"ContainerStarted","Data":"fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297"}
Jul 22 18:50:42.846360 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:42.846351    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt" podUID="95f18735-294f-44fe-8004-dbbcd6b71402" containerName="registry-server" containerID="cri-o://fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297" gracePeriod=2
Jul 22 18:50:42.847812 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:42.847788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h4qpd" event={"ID":"36fac68c-df61-4246-b37c-66c8772335dc","Type":"ContainerStarted","Data":"8f3362dafce0b5f7703600cc107d088c7f473709d9ac725e9e2f9a654e46646f"}
Jul 22 18:50:42.864680 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:42.864591    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt" podStartSLOduration=1.75296659 podStartE2EDuration="3.86456809s" totalImagesPullingTime="2.1116015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:50:42.863352868 +0000 UTC m=+643.753893801" watchObservedRunningTime="2026-07-22 18:50:42.86456809 +0000 UTC m=+643.755109052"
Jul 22 18:50:42.880193 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:42.880154    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h4qpd" podStartSLOduration=2.327031668 podStartE2EDuration="3.880136144s" totalImagesPullingTime="1.553104476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:50:42.879532378 +0000 UTC m=+643.770073312" watchObservedRunningTime="2026-07-22 18:50:42.880136144 +0000 UTC m=+643.770677073"
Jul 22 18:50:43.087135 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.087115    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt"
Jul 22 18:50:43.140353 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.140323    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbff5\" (UniqueName: \"kubernetes.io/projected/95f18735-294f-44fe-8004-dbbcd6b71402-kube-api-access-kbff5\") pod \"95f18735-294f-44fe-8004-dbbcd6b71402\" (UID: \"95f18735-294f-44fe-8004-dbbcd6b71402\") "
Jul 22 18:50:43.142393 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.142368    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95f18735-294f-44fe-8004-dbbcd6b71402-kube-api-access-kbff5" (OuterVolumeSpecName: "kube-api-access-kbff5") pod "95f18735-294f-44fe-8004-dbbcd6b71402" (UID: "95f18735-294f-44fe-8004-dbbcd6b71402"). InnerVolumeSpecName "kube-api-access-kbff5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:50:43.241084 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.241056    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbff5\" (UniqueName: \"kubernetes.io/projected/95f18735-294f-44fe-8004-dbbcd6b71402-kube-api-access-kbff5\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:50:43.853442 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.853364    2576 generic.go:356] "Generic (PLEG): container finished" podID="95f18735-294f-44fe-8004-dbbcd6b71402" containerID="fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297" exitCode=0
Jul 22 18:50:43.853442 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.853400    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt"
Jul 22 18:50:43.853924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.853447    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt" event={"ID":"95f18735-294f-44fe-8004-dbbcd6b71402","Type":"ContainerDied","Data":"fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297"}
Jul 22 18:50:43.853924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.853498    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7nrkt" event={"ID":"95f18735-294f-44fe-8004-dbbcd6b71402","Type":"ContainerDied","Data":"25ff64dab3d65f4c598b472ad53179c28d9bcbc324a922871248c939e253ea5e"}
Jul 22 18:50:43.853924 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.853519    2576 scope.go:117] "RemoveContainer" containerID="fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297"
Jul 22 18:50:43.861774 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.861757    2576 scope.go:117] "RemoveContainer" containerID="fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297"
Jul 22 18:50:43.862027 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:50:43.862005    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297\": container with ID starting with fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297 not found: ID does not exist" containerID="fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297"
Jul 22 18:50:43.862091 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.862034    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297"} err="failed to get container status \"fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297\": rpc error: code = NotFound desc = could not find container \"fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297\": container with ID starting with fad794e029f313ccc03df7f165b497a4b99865633d433a361eb1bc7a15d86297 not found: ID does not exist"
Jul 22 18:50:43.869402 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.869381    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nrkt"]
Jul 22 18:50:43.871271 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:43.871253    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7nrkt"]
Jul 22 18:50:45.687336 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:45.687295    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95f18735-294f-44fe-8004-dbbcd6b71402" path="/var/lib/kubelet/pods/95f18735-294f-44fe-8004-dbbcd6b71402/volumes"
Jul 22 18:50:50.199651 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:50.199601    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-h4qpd"
Jul 22 18:50:50.200070 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:50.199713    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-h4qpd"
Jul 22 18:50:50.222026 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:50.222000    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-h4qpd"
Jul 22 18:50:50.906474 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:50:50.906441    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-h4qpd"
Jul 22 18:51:11.549098 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.549065    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"]
Jul 22 18:51:11.549556 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.549343    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95f18735-294f-44fe-8004-dbbcd6b71402" containerName="registry-server"
Jul 22 18:51:11.549556 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.549353    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95f18735-294f-44fe-8004-dbbcd6b71402" containerName="registry-server"
Jul 22 18:51:11.549556 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.549406    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="95f18735-294f-44fe-8004-dbbcd6b71402" containerName="registry-server"
Jul 22 18:51:11.554010 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.553988    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"
Jul 22 18:51:11.556586 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.556554    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-jcgqb\""
Jul 22 18:51:11.567393 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.567370    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"]
Jul 22 18:51:11.649028 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.649002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfsgh\" (UniqueName: \"kubernetes.io/projected/5e9062bc-e607-4431-bda2-cb820b89c98d-kube-api-access-vfsgh\") pod \"authorino-operator-75cc8d5cb7-rjl2x\" (UID: \"5e9062bc-e607-4431-bda2-cb820b89c98d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"
Jul 22 18:51:11.750330 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.750302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfsgh\" (UniqueName: \"kubernetes.io/projected/5e9062bc-e607-4431-bda2-cb820b89c98d-kube-api-access-vfsgh\") pod \"authorino-operator-75cc8d5cb7-rjl2x\" (UID: \"5e9062bc-e607-4431-bda2-cb820b89c98d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"
Jul 22 18:51:11.762634 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.762608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfsgh\" (UniqueName: \"kubernetes.io/projected/5e9062bc-e607-4431-bda2-cb820b89c98d-kube-api-access-vfsgh\") pod \"authorino-operator-75cc8d5cb7-rjl2x\" (UID: \"5e9062bc-e607-4431-bda2-cb820b89c98d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"
Jul 22 18:51:11.863841 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.863781    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"
Jul 22 18:51:11.991490 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:11.991462    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"]
Jul 22 18:51:11.994861 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:51:11.994831    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e9062bc_e607_4431_bda2_cb820b89c98d.slice/crio-7eb89290374442a2a621487ffb445b4836d49a716af15b687dc2dbd3a5665f21 WatchSource:0}: Error finding container 7eb89290374442a2a621487ffb445b4836d49a716af15b687dc2dbd3a5665f21: Status 404 returned error can't find the container with id 7eb89290374442a2a621487ffb445b4836d49a716af15b687dc2dbd3a5665f21
Jul 22 18:51:12.970533 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:12.970495    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x" event={"ID":"5e9062bc-e607-4431-bda2-cb820b89c98d","Type":"ContainerStarted","Data":"7eb89290374442a2a621487ffb445b4836d49a716af15b687dc2dbd3a5665f21"}
Jul 22 18:51:14.337020 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.336991    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"]
Jul 22 18:51:14.340493 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.340470    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:14.343330 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.343309    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-twj8b\""
Jul 22 18:51:14.349431 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.349411    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"]
Jul 22 18:51:14.473225 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.473194    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzxj4\" (UniqueName: \"kubernetes.io/projected/84108f23-5003-43bf-bbd9-4db3debbcfbd-kube-api-access-hzxj4\") pod \"limitador-operator-controller-manager-859d7b78bd-hflkx\" (UID: \"84108f23-5003-43bf-bbd9-4db3debbcfbd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:14.574565 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.574529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzxj4\" (UniqueName: \"kubernetes.io/projected/84108f23-5003-43bf-bbd9-4db3debbcfbd-kube-api-access-hzxj4\") pod \"limitador-operator-controller-manager-859d7b78bd-hflkx\" (UID: \"84108f23-5003-43bf-bbd9-4db3debbcfbd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:14.585321 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.585292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzxj4\" (UniqueName: \"kubernetes.io/projected/84108f23-5003-43bf-bbd9-4db3debbcfbd-kube-api-access-hzxj4\") pod \"limitador-operator-controller-manager-859d7b78bd-hflkx\" (UID: \"84108f23-5003-43bf-bbd9-4db3debbcfbd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:14.650956 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.650929    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:14.779480 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.779453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"]
Jul 22 18:51:14.782015 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:51:14.781985    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84108f23_5003_43bf_bbd9_4db3debbcfbd.slice/crio-438c23b3219e78b38c26b86309493bed5058327ebfacac62f92d2d242ff59eb2 WatchSource:0}: Error finding container 438c23b3219e78b38c26b86309493bed5058327ebfacac62f92d2d242ff59eb2: Status 404 returned error can't find the container with id 438c23b3219e78b38c26b86309493bed5058327ebfacac62f92d2d242ff59eb2
Jul 22 18:51:14.981210 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.981122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx" event={"ID":"84108f23-5003-43bf-bbd9-4db3debbcfbd","Type":"ContainerStarted","Data":"438c23b3219e78b38c26b86309493bed5058327ebfacac62f92d2d242ff59eb2"}
Jul 22 18:51:14.982545 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.982520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x" event={"ID":"5e9062bc-e607-4431-bda2-cb820b89c98d","Type":"ContainerStarted","Data":"33287c0a194cc46bfb78ffea9b082a7c958a35466bc9f31cbb40b9143eb3f393"}
Jul 22 18:51:14.982685 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:14.982670    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"
Jul 22 18:51:15.000701 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:15.000529    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x" podStartSLOduration=1.612404121 podStartE2EDuration="4.00051363s" totalImagesPullingTime="2.388109509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:14.999936228 +0000 UTC m=+675.890477159" watchObservedRunningTime="2026-07-22 18:51:15.00051363 +0000 UTC m=+675.891054580"
Jul 22 18:51:16.995240 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:16.995160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx" event={"ID":"84108f23-5003-43bf-bbd9-4db3debbcfbd","Type":"ContainerStarted","Data":"b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c"}
Jul 22 18:51:16.995240 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:16.995228    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:17.017730 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.017685    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx" podStartSLOduration=1.0750192140000001 podStartE2EDuration="3.01766603s" totalImagesPullingTime="1.942646816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:17.016072512 +0000 UTC m=+677.906613447" watchObservedRunningTime="2026-07-22 18:51:17.01766603 +0000 UTC m=+677.908207032"
Jul 22 18:51:17.071704 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.071676    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"]
Jul 22 18:51:17.075247 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.075228    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"
Jul 22 18:51:17.078330 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.078311    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qs56r\""
Jul 22 18:51:17.078442 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.078308    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 22 18:51:17.087123 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.087102    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"]
Jul 22 18:51:17.195348 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.195318    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlzwz\" (UniqueName: \"kubernetes.io/projected/cdb8da78-da94-4dce-b767-d65cd67512ec-kube-api-access-hlzwz\") pod \"dns-operator-controller-manager-864dd78c4c-wkjqb\" (UID: \"cdb8da78-da94-4dce-b767-d65cd67512ec\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"
Jul 22 18:51:17.296144 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.296075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlzwz\" (UniqueName: \"kubernetes.io/projected/cdb8da78-da94-4dce-b767-d65cd67512ec-kube-api-access-hlzwz\") pod \"dns-operator-controller-manager-864dd78c4c-wkjqb\" (UID: \"cdb8da78-da94-4dce-b767-d65cd67512ec\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"
Jul 22 18:51:17.307419 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.307391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlzwz\" (UniqueName: \"kubernetes.io/projected/cdb8da78-da94-4dce-b767-d65cd67512ec-kube-api-access-hlzwz\") pod \"dns-operator-controller-manager-864dd78c4c-wkjqb\" (UID: \"cdb8da78-da94-4dce-b767-d65cd67512ec\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"
Jul 22 18:51:17.385509 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.385482    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"
Jul 22 18:51:17.509692 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:17.509612    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"]
Jul 22 18:51:17.511974 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:51:17.511946    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdb8da78_da94_4dce_b767_d65cd67512ec.slice/crio-f79c83a90ec1c74bbf5063141b852ac554e604c64fe9854c2d70a5e3ab53e207 WatchSource:0}: Error finding container f79c83a90ec1c74bbf5063141b852ac554e604c64fe9854c2d70a5e3ab53e207: Status 404 returned error can't find the container with id f79c83a90ec1c74bbf5063141b852ac554e604c64fe9854c2d70a5e3ab53e207
Jul 22 18:51:18.000412 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:18.000381    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb" event={"ID":"cdb8da78-da94-4dce-b767-d65cd67512ec","Type":"ContainerStarted","Data":"f79c83a90ec1c74bbf5063141b852ac554e604c64fe9854c2d70a5e3ab53e207"}
Jul 22 18:51:20.009650 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:20.009597    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb" event={"ID":"cdb8da78-da94-4dce-b767-d65cd67512ec","Type":"ContainerStarted","Data":"8849cc507556e1e42be06854a44df2bbeca40bede7b490fdc1cb4817765cd2c6"}
Jul 22 18:51:20.010142 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:20.009750    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"
Jul 22 18:51:20.028544 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:20.028499    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb" podStartSLOduration=0.733373528 podStartE2EDuration="3.028486932s" totalImagesPullingTime="2.295113404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:20.026478445 +0000 UTC m=+680.917019378" watchObservedRunningTime="2026-07-22 18:51:20.028486932 +0000 UTC m=+680.919027864"
Jul 22 18:51:21.651082 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.651048    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"]
Jul 22 18:51:21.654384 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.654367    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:21.658150 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.658127    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rn8nc\""
Jul 22 18:51:21.671300 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.671276    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"]
Jul 22 18:51:21.731710 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.731678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efe36027-b8ae-4ed6-b444-041799350b4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" (UID: \"efe36027-b8ae-4ed6-b444-041799350b4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:21.731831 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.731791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f9tv\" (UniqueName: \"kubernetes.io/projected/efe36027-b8ae-4ed6-b444-041799350b4f-kube-api-access-7f9tv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" (UID: \"efe36027-b8ae-4ed6-b444-041799350b4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:21.832655 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.832614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efe36027-b8ae-4ed6-b444-041799350b4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" (UID: \"efe36027-b8ae-4ed6-b444-041799350b4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:21.832785 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.832769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9tv\" (UniqueName: \"kubernetes.io/projected/efe36027-b8ae-4ed6-b444-041799350b4f-kube-api-access-7f9tv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" (UID: \"efe36027-b8ae-4ed6-b444-041799350b4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:21.833022 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.832995    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efe36027-b8ae-4ed6-b444-041799350b4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" (UID: \"efe36027-b8ae-4ed6-b444-041799350b4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:21.841942 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.841924    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f9tv\" (UniqueName: \"kubernetes.io/projected/efe36027-b8ae-4ed6-b444-041799350b4f-kube-api-access-7f9tv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" (UID: \"efe36027-b8ae-4ed6-b444-041799350b4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:21.963652 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:21.963570    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:22.091870 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:22.091847    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"]
Jul 22 18:51:22.094228 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:51:22.094204    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe36027_b8ae_4ed6_b444_041799350b4f.slice/crio-e981b33782283edcf3940286a28cd2b87f9ebc1d318c8812f9e9b57fec3487f7 WatchSource:0}: Error finding container e981b33782283edcf3940286a28cd2b87f9ebc1d318c8812f9e9b57fec3487f7: Status 404 returned error can't find the container with id e981b33782283edcf3940286a28cd2b87f9ebc1d318c8812f9e9b57fec3487f7
Jul 22 18:51:23.023083 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:23.023050    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs" event={"ID":"efe36027-b8ae-4ed6-b444-041799350b4f","Type":"ContainerStarted","Data":"e981b33782283edcf3940286a28cd2b87f9ebc1d318c8812f9e9b57fec3487f7"}
Jul 22 18:51:25.991791 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:25.991752    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjl2x"
Jul 22 18:51:28.003704 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:28.003668    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:29.050572 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:29.050539    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs" event={"ID":"efe36027-b8ae-4ed6-b444-041799350b4f","Type":"ContainerStarted","Data":"07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3"}
Jul 22 18:51:29.050954 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:29.050614    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:29.075865 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:29.075818    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs" podStartSLOduration=1.832291196 podStartE2EDuration="8.075803668s" totalImagesPullingTime="6.243512472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:29.073044856 +0000 UTC m=+689.963585790" watchObservedRunningTime="2026-07-22 18:51:29.075803668 +0000 UTC m=+689.966344600"
Jul 22 18:51:31.015054 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.015026    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wkjqb"
Jul 22 18:51:31.187065 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.187030    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"]
Jul 22 18:51:31.190551 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.190534    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.193808 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.193776    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 22 18:51:31.194226 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.194198    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8lbcw\""
Jul 22 18:51:31.194742 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.194727    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 22 18:51:31.200497 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.200475    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"]
Jul 22 18:51:31.311798 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.311717    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/696d68e9-7f67-4900-afa2-03c26e88863f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.311798 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.311766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg2f6\" (UniqueName: \"kubernetes.io/projected/696d68e9-7f67-4900-afa2-03c26e88863f-kube-api-access-mg2f6\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.311970 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.311823    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/696d68e9-7f67-4900-afa2-03c26e88863f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.412826 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.412792    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/696d68e9-7f67-4900-afa2-03c26e88863f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.412992 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.412838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg2f6\" (UniqueName: \"kubernetes.io/projected/696d68e9-7f67-4900-afa2-03c26e88863f-kube-api-access-mg2f6\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.412992 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.412868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/696d68e9-7f67-4900-afa2-03c26e88863f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.413109 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:31.412992    2576 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 22 18:51:31.413109 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:31.413076    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/696d68e9-7f67-4900-afa2-03c26e88863f-plugin-serving-cert podName:696d68e9-7f67-4900-afa2-03c26e88863f nodeName:}" failed. No retries permitted until 2026-07-22 18:51:31.913053917 +0000 UTC m=+692.803594830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/696d68e9-7f67-4900-afa2-03c26e88863f-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-w7rbs" (UID: "696d68e9-7f67-4900-afa2-03c26e88863f") : secret "plugin-serving-cert" not found
Jul 22 18:51:31.413492 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.413472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/696d68e9-7f67-4900-afa2-03c26e88863f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.429614 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.429587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg2f6\" (UniqueName: \"kubernetes.io/projected/696d68e9-7f67-4900-afa2-03c26e88863f-kube-api-access-mg2f6\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.916087 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.916051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/696d68e9-7f67-4900-afa2-03c26e88863f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:31.918395 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:31.918363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/696d68e9-7f67-4900-afa2-03c26e88863f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w7rbs\" (UID: \"696d68e9-7f67-4900-afa2-03c26e88863f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:32.099580 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:32.099548    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"
Jul 22 18:51:32.232667 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:32.232640    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs"]
Jul 22 18:51:32.234806 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:51:32.234771    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod696d68e9_7f67_4900_afa2_03c26e88863f.slice/crio-ff288597e136f06d112c24f5f30ad988f05d7a81243c9c272e99d8ebf63e01bb WatchSource:0}: Error finding container ff288597e136f06d112c24f5f30ad988f05d7a81243c9c272e99d8ebf63e01bb: Status 404 returned error can't find the container with id ff288597e136f06d112c24f5f30ad988f05d7a81243c9c272e99d8ebf63e01bb
Jul 22 18:51:33.070499 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:33.070459    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs" event={"ID":"696d68e9-7f67-4900-afa2-03c26e88863f","Type":"ContainerStarted","Data":"ff288597e136f06d112c24f5f30ad988f05d7a81243c9c272e99d8ebf63e01bb"}
Jul 22 18:51:40.056905 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:40.056871    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:41.147986 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.147950    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"]
Jul 22 18:51:41.153211 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.153189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:41.170487 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.170457    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"]
Jul 22 18:51:41.296133 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.296093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" (UID: \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:41.296314 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.296153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27bsf\" (UniqueName: \"kubernetes.io/projected/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-kube-api-access-27bsf\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" (UID: \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:41.397573 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.397521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" (UID: \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:41.397773 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.397598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27bsf\" (UniqueName: \"kubernetes.io/projected/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-kube-api-access-27bsf\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" (UID: \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:41.398028 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.397970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" (UID: \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:41.412729 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.412702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27bsf\" (UniqueName: \"kubernetes.io/projected/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-kube-api-access-27bsf\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" (UID: \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:41.465238 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:41.465209    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:42.138399 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.138364    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"]
Jul 22 18:51:42.141036 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.141007    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"]
Jul 22 18:51:42.150127 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.150098    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"]
Jul 22 18:51:42.150512 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.150308    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" containerName="manager" containerID="cri-o://07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3" gracePeriod=2
Jul 22 18:51:42.161131 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.161076    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"]
Jul 22 18:51:42.161612 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:42.161568    2576 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"efe36027-b8ae-4ed6-b444-041799350b4f\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\": pods \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:42.163385 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.163359    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"]
Jul 22 18:51:42.163839 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.163820    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" containerName="manager"
Jul 22 18:51:42.163974 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.163956    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" containerName="manager"
Jul 22 18:51:42.164103 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.164053    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" containerName="manager"
Jul 22 18:51:42.167869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.167849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:51:42.172550 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:42.170755    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:42.172550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.172011    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"]
Jul 22 18:51:42.172550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.172195    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx" podUID="84108f23-5003-43bf-bbd9-4db3debbcfbd" containerName="manager" containerID="cri-o://b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c" gracePeriod=2
Jul 22 18:51:42.175464 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.175439    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"]
Jul 22 18:51:42.180550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.180295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:51:42.180795 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.180096    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"]
Jul 22 18:51:42.187378 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.187358    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"]
Jul 22 18:51:42.193968 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.193928    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"]
Jul 22 18:51:42.206474 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:42.203004    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:42.234113 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:42.234063    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflkx\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="84108f23-5003-43bf-bbd9-4db3debbcfbd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:42.308324 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.308295    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn6cd\" (UniqueName: \"kubernetes.io/projected/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-kube-api-access-kn6cd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cmb7l\" (UID: \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:51:42.308549 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.308340    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8jkmc\" (UID: \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:51:42.308549 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.308373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cmb7l\" (UID: \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:51:42.308549 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.308488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh2gn\" (UniqueName: \"kubernetes.io/projected/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-kube-api-access-xh2gn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8jkmc\" (UID: \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:51:42.409869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.409778    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn6cd\" (UniqueName: \"kubernetes.io/projected/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-kube-api-access-kn6cd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cmb7l\" (UID: \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:51:42.409869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.409821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8jkmc\" (UID: \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:51:42.409869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.409849    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cmb7l\" (UID: \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:51:42.410146 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.409880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh2gn\" (UniqueName: \"kubernetes.io/projected/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-kube-api-access-xh2gn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8jkmc\" (UID: \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:51:42.410292 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.410262    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8jkmc\" (UID: \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:51:42.410292 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.410256    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cmb7l\" (UID: \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:51:42.423663 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.423619    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn6cd\" (UniqueName: \"kubernetes.io/projected/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-kube-api-access-kn6cd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cmb7l\" (UID: \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:51:42.423663 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.423656    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh2gn\" (UniqueName: \"kubernetes.io/projected/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-kube-api-access-xh2gn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8jkmc\" (UID: \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:51:42.569034 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.568996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:51:42.576159 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:42.576131    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:51:55.025385 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.025354    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:55.027868 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:55.027840    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:55.041062 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.041035    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:55.043599 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:55.043563    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:55.045834 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:55.045800    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflkx\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="84108f23-5003-43bf-bbd9-4db3debbcfbd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:55.049772 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.049749    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"]
Jul 22 18:51:55.052359 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:51:55.052330    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3edca8_b7c7_4444_89c2_46dd0dc163ae.slice/crio-dc1537e9c35400f6df464e3c2fc5877d93f5e5a98a7115ecc166be0c10d6d9ab WatchSource:0}: Error finding container dc1537e9c35400f6df464e3c2fc5877d93f5e5a98a7115ecc166be0c10d6d9ab: Status 404 returned error can't find the container with id dc1537e9c35400f6df464e3c2fc5877d93f5e5a98a7115ecc166be0c10d6d9ab
Jul 22 18:51:55.076026 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.076002    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"]
Jul 22 18:51:55.085703 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:51:55.085682    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1233c0d7_3813_4f5e_9352_dfef0cd6dc02.slice/crio-1ccbcd1df7c80d7ef8622ab149c02a3f678536f537eed58f8f87ff8318e10471 WatchSource:0}: Error finding container 1ccbcd1df7c80d7ef8622ab149c02a3f678536f537eed58f8f87ff8318e10471: Status 404 returned error can't find the container with id 1ccbcd1df7c80d7ef8622ab149c02a3f678536f537eed58f8f87ff8318e10471
Jul 22 18:51:55.119789 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.119764    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzxj4\" (UniqueName: \"kubernetes.io/projected/84108f23-5003-43bf-bbd9-4db3debbcfbd-kube-api-access-hzxj4\") pod \"84108f23-5003-43bf-bbd9-4db3debbcfbd\" (UID: \"84108f23-5003-43bf-bbd9-4db3debbcfbd\") "
Jul 22 18:51:55.119880 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.119837    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f9tv\" (UniqueName: \"kubernetes.io/projected/efe36027-b8ae-4ed6-b444-041799350b4f-kube-api-access-7f9tv\") pod \"efe36027-b8ae-4ed6-b444-041799350b4f\" (UID: \"efe36027-b8ae-4ed6-b444-041799350b4f\") "
Jul 22 18:51:55.119946 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.119931    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efe36027-b8ae-4ed6-b444-041799350b4f-extensions-socket-volume\") pod \"efe36027-b8ae-4ed6-b444-041799350b4f\" (UID: \"efe36027-b8ae-4ed6-b444-041799350b4f\") "
Jul 22 18:51:55.120453 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.120424    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe36027-b8ae-4ed6-b444-041799350b4f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "efe36027-b8ae-4ed6-b444-041799350b4f" (UID: "efe36027-b8ae-4ed6-b444-041799350b4f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 18:51:55.121575 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.121548    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84108f23-5003-43bf-bbd9-4db3debbcfbd-kube-api-access-hzxj4" (OuterVolumeSpecName: "kube-api-access-hzxj4") pod "84108f23-5003-43bf-bbd9-4db3debbcfbd" (UID: "84108f23-5003-43bf-bbd9-4db3debbcfbd"). InnerVolumeSpecName "kube-api-access-hzxj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:51:55.121790 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.121774    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe36027-b8ae-4ed6-b444-041799350b4f-kube-api-access-7f9tv" (OuterVolumeSpecName: "kube-api-access-7f9tv") pod "efe36027-b8ae-4ed6-b444-041799350b4f" (UID: "efe36027-b8ae-4ed6-b444-041799350b4f"). InnerVolumeSpecName "kube-api-access-7f9tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:51:55.179049 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.179014    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc" event={"ID":"1233c0d7-3813-4f5e-9352-dfef0cd6dc02","Type":"ContainerStarted","Data":"93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34"}
Jul 22 18:51:55.179221 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.179079    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc" event={"ID":"1233c0d7-3813-4f5e-9352-dfef0cd6dc02","Type":"ContainerStarted","Data":"1ccbcd1df7c80d7ef8622ab149c02a3f678536f537eed58f8f87ff8318e10471"}
Jul 22 18:51:55.179221 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.179131    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:51:55.180691 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.180664    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l" event={"ID":"8f3edca8-b7c7-4444-89c2-46dd0dc163ae","Type":"ContainerStarted","Data":"97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb"}
Jul 22 18:51:55.180831 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.180713    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l" event={"ID":"8f3edca8-b7c7-4444-89c2-46dd0dc163ae","Type":"ContainerStarted","Data":"dc1537e9c35400f6df464e3c2fc5877d93f5e5a98a7115ecc166be0c10d6d9ab"}
Jul 22 18:51:55.180831 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.180743    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:51:55.181707 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:55.181674    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:55.182146 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.182128    2576 generic.go:356] "Generic (PLEG): container finished" podID="efe36027-b8ae-4ed6-b444-041799350b4f" containerID="07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3" exitCode=0
Jul 22 18:51:55.182195 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.182162    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:55.182237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.182199    2576 scope.go:117] "RemoveContainer" containerID="07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3"
Jul 22 18:51:55.183740 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.183712    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs" event={"ID":"696d68e9-7f67-4900-afa2-03c26e88863f","Type":"ContainerStarted","Data":"7dd94285e3e12f042f9d30c1ed43d68bd4ea5c2cd80a5adaf5d2da4be146c352"}
Jul 22 18:51:55.185088 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.185062    2576 generic.go:356] "Generic (PLEG): container finished" podID="84108f23-5003-43bf-bbd9-4db3debbcfbd" containerID="b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c" exitCode=0
Jul 22 18:51:55.185209 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.185092    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:55.190933 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.190909    2576 scope.go:117] "RemoveContainer" containerID="07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3"
Jul 22 18:51:55.191204 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:55.191181    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3\": container with ID starting with 07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3 not found: ID does not exist" containerID="07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3"
Jul 22 18:51:55.191307 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.191211    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3"} err="failed to get container status \"07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3\": rpc error: code = NotFound desc = could not find container \"07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3\": container with ID starting with 07b520272af9f380991135a74ccca39935b8ef8ea73650ba00e94814bcdf43c3 not found: ID does not exist"
Jul 22 18:51:55.191307 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.191233    2576 scope.go:117] "RemoveContainer" containerID="b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c"
Jul 22 18:51:55.199085 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.199069    2576 scope.go:117] "RemoveContainer" containerID="b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c"
Jul 22 18:51:55.199311 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:55.199294    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c\": container with ID starting with b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c not found: ID does not exist" containerID="b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c"
Jul 22 18:51:55.199355 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.199317    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c"} err="failed to get container status \"b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c\": rpc error: code = NotFound desc = could not find container \"b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c\": container with ID starting with b32a441ae53c21f58f93e5cf1199861798749f90454f667e7735d80ee2b6118c not found: ID does not exist"
Jul 22 18:51:55.208791 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.208744    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc" podStartSLOduration=13.208728526 podStartE2EDuration="13.208728526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:55.208546016 +0000 UTC m=+716.099086950" watchObservedRunningTime="2026-07-22 18:51:55.208728526 +0000 UTC m=+716.099269453"
Jul 22 18:51:55.211310 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:55.211283    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflkx\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="84108f23-5003-43bf-bbd9-4db3debbcfbd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:55.213702 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:55.213680    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hflkx\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="84108f23-5003-43bf-bbd9-4db3debbcfbd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hflkx"
Jul 22 18:51:55.221257 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.221234    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzxj4\" (UniqueName: \"kubernetes.io/projected/84108f23-5003-43bf-bbd9-4db3debbcfbd-kube-api-access-hzxj4\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:51:55.221391 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.221262    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f9tv\" (UniqueName: \"kubernetes.io/projected/efe36027-b8ae-4ed6-b444-041799350b4f-kube-api-access-7f9tv\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:51:55.221391 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.221279    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/efe36027-b8ae-4ed6-b444-041799350b4f-extensions-socket-volume\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:51:55.237282 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.237243    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7rbs" podStartSLOduration=1.535014334 podStartE2EDuration="24.23722951s" totalImagesPullingTime="22.702215176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:55.236026233 +0000 UTC m=+716.126567165" watchObservedRunningTime="2026-07-22 18:51:55.23722951 +0000 UTC m=+716.127770441"
Jul 22 18:51:55.239131 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:55.239106    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c6cqs\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c6cqs"
Jul 22 18:51:55.261659 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.261587    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l" podStartSLOduration=13.261567878 podStartE2EDuration="13.261567878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:55.259893841 +0000 UTC m=+716.150434775" watchObservedRunningTime="2026-07-22 18:51:55.261567878 +0000 UTC m=+716.152108812"
Jul 22 18:51:55.687858 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.687828    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84108f23-5003-43bf-bbd9-4db3debbcfbd" path="/var/lib/kubelet/pods/84108f23-5003-43bf-bbd9-4db3debbcfbd/volumes"
Jul 22 18:51:55.688212 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:55.688199    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe36027-b8ae-4ed6-b444-041799350b4f" path="/var/lib/kubelet/pods/efe36027-b8ae-4ed6-b444-041799350b4f/volumes"
Jul 22 18:51:57.420915 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:57.420877    2576 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-vts4r_kuadrant-system_d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2_0(a56cd6aa6a6e0f9139bc11b93aed2ccd4107239c5f1ff631678b3a316fbacc48): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-vts4r to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 22 18:51:57.421182 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:57.420940    2576 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-vts4r_kuadrant-system_d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2_0(a56cd6aa6a6e0f9139bc11b93aed2ccd4107239c5f1ff631678b3a316fbacc48): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-vts4r to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:58.200535 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:58.200508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:58.202983 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:58.202952    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:58.206935 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:58.206916    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:58.209345 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:58.209319    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:58.243721 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:58.243699    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27bsf\" (UniqueName: \"kubernetes.io/projected/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-kube-api-access-27bsf\") pod \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\" (UID: \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\") "
Jul 22 18:51:58.243791 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:58.243744    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-extensions-socket-volume\") pod \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\" (UID: \"d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2\") "
Jul 22 18:51:58.244041 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:58.244022    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2" (UID: "d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 18:51:58.245876 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:58.245857    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-kube-api-access-27bsf" (OuterVolumeSpecName: "kube-api-access-27bsf") pod "d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2" (UID: "d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2"). InnerVolumeSpecName "kube-api-access-27bsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:51:58.344178 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:58.344154    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27bsf\" (UniqueName: \"kubernetes.io/projected/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-kube-api-access-27bsf\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:51:58.344178 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:58.344174    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2-extensions-socket-volume\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:51:59.204279 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:59.204247    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:59.206823 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:59.206788    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:59.212306 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:59.212279    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:59.686684 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:51:59.686617    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vts4r\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vts4r"
Jul 22 18:51:59.688539 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:51:59.688514    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2" path="/var/lib/kubelet/pods/d2e9ee27-6e81-49b9-9ba8-9d26a8b1c6d2/volumes"
Jul 22 18:52:06.193418 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.193385    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:52:06.193847 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.193459    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:52:06.276234 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.276197    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"]
Jul 22 18:52:06.276461 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.276408    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc" podUID="1233c0d7-3813-4f5e-9352-dfef0cd6dc02" containerName="manager" containerID="cri-o://93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34" gracePeriod=10
Jul 22 18:52:06.520314 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.520293    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:52:06.560365 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.560337    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"]
Jul 22 18:52:06.560687 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.560673    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1233c0d7-3813-4f5e-9352-dfef0cd6dc02" containerName="manager"
Jul 22 18:52:06.560736 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.560689    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1233c0d7-3813-4f5e-9352-dfef0cd6dc02" containerName="manager"
Jul 22 18:52:06.560736 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.560704    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84108f23-5003-43bf-bbd9-4db3debbcfbd" containerName="manager"
Jul 22 18:52:06.560736 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.560709    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="84108f23-5003-43bf-bbd9-4db3debbcfbd" containerName="manager"
Jul 22 18:52:06.560824 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.560757    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1233c0d7-3813-4f5e-9352-dfef0cd6dc02" containerName="manager"
Jul 22 18:52:06.560824 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.560767    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="84108f23-5003-43bf-bbd9-4db3debbcfbd" containerName="manager"
Jul 22 18:52:06.562706 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.562672    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:06.583801 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.583780    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"]
Jul 22 18:52:06.608148 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.608123    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh2gn\" (UniqueName: \"kubernetes.io/projected/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-kube-api-access-xh2gn\") pod \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\" (UID: \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\") "
Jul 22 18:52:06.608232 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.608195    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-extensions-socket-volume\") pod \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\" (UID: \"1233c0d7-3813-4f5e-9352-dfef0cd6dc02\") "
Jul 22 18:52:06.608399 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.608379    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmbwd\" (UniqueName: \"kubernetes.io/projected/ca648b81-0fef-4dc5-a239-88b70c12dd8b-kube-api-access-xmbwd\") pod \"kuadrant-operator-controller-manager-86878c769f-w2rqc\" (UID: \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:06.608462 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.608444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca648b81-0fef-4dc5-a239-88b70c12dd8b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w2rqc\" (UID: \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:06.608586 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.608552    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1233c0d7-3813-4f5e-9352-dfef0cd6dc02" (UID: "1233c0d7-3813-4f5e-9352-dfef0cd6dc02"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 18:52:06.610664 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.610620    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-kube-api-access-xh2gn" (OuterVolumeSpecName: "kube-api-access-xh2gn") pod "1233c0d7-3813-4f5e-9352-dfef0cd6dc02" (UID: "1233c0d7-3813-4f5e-9352-dfef0cd6dc02"). InnerVolumeSpecName "kube-api-access-xh2gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:52:06.708898 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.708871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca648b81-0fef-4dc5-a239-88b70c12dd8b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w2rqc\" (UID: \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:06.709042 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.708930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmbwd\" (UniqueName: \"kubernetes.io/projected/ca648b81-0fef-4dc5-a239-88b70c12dd8b-kube-api-access-xmbwd\") pod \"kuadrant-operator-controller-manager-86878c769f-w2rqc\" (UID: \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:06.709042 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.708970    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh2gn\" (UniqueName: \"kubernetes.io/projected/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-kube-api-access-xh2gn\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:52:06.709042 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.708979    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1233c0d7-3813-4f5e-9352-dfef0cd6dc02-extensions-socket-volume\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:52:06.709305 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.709285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca648b81-0fef-4dc5-a239-88b70c12dd8b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w2rqc\" (UID: \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:06.717949 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.717921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmbwd\" (UniqueName: \"kubernetes.io/projected/ca648b81-0fef-4dc5-a239-88b70c12dd8b-kube-api-access-xmbwd\") pod \"kuadrant-operator-controller-manager-86878c769f-w2rqc\" (UID: \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:06.873575 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:06.873544    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:07.003045 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.003019    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"]
Jul 22 18:52:07.005405 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:52:07.005378    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca648b81_0fef_4dc5_a239_88b70c12dd8b.slice/crio-c271d518b63d848661dfaa04fa207033337d6b91b8d3f0a5a3fa619316b62738 WatchSource:0}: Error finding container c271d518b63d848661dfaa04fa207033337d6b91b8d3f0a5a3fa619316b62738: Status 404 returned error can't find the container with id c271d518b63d848661dfaa04fa207033337d6b91b8d3f0a5a3fa619316b62738
Jul 22 18:52:07.239696 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.239605    2576 generic.go:356] "Generic (PLEG): container finished" podID="1233c0d7-3813-4f5e-9352-dfef0cd6dc02" containerID="93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34" exitCode=0
Jul 22 18:52:07.239696 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.239664    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"
Jul 22 18:52:07.240154 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.239661    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc" event={"ID":"1233c0d7-3813-4f5e-9352-dfef0cd6dc02","Type":"ContainerDied","Data":"93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34"}
Jul 22 18:52:07.240154 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.239792    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc" event={"ID":"1233c0d7-3813-4f5e-9352-dfef0cd6dc02","Type":"ContainerDied","Data":"1ccbcd1df7c80d7ef8622ab149c02a3f678536f537eed58f8f87ff8318e10471"}
Jul 22 18:52:07.240154 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.239825    2576 scope.go:117] "RemoveContainer" containerID="93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34"
Jul 22 18:52:07.241377 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.241344    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc" event={"ID":"ca648b81-0fef-4dc5-a239-88b70c12dd8b","Type":"ContainerStarted","Data":"987453b707058432eed159c4c0c1573afec30852f9af3e81a0e2683fb644ca97"}
Jul 22 18:52:07.241479 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.241392    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc" event={"ID":"ca648b81-0fef-4dc5-a239-88b70c12dd8b","Type":"ContainerStarted","Data":"c271d518b63d848661dfaa04fa207033337d6b91b8d3f0a5a3fa619316b62738"}
Jul 22 18:52:07.241479 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.241422    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:07.249206 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.249182    2576 scope.go:117] "RemoveContainer" containerID="93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34"
Jul 22 18:52:07.249479 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:52:07.249453    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34\": container with ID starting with 93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34 not found: ID does not exist" containerID="93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34"
Jul 22 18:52:07.249567 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.249487    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34"} err="failed to get container status \"93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34\": rpc error: code = NotFound desc = could not find container \"93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34\": container with ID starting with 93f4a1e799b03e54fd123f84a95457b664435f4837f6dcc5434e72adc06b2a34 not found: ID does not exist"
Jul 22 18:52:07.264353 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.264306    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc" podStartSLOduration=1.264293071 podStartE2EDuration="1.264293071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:52:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:07.262409315 +0000 UTC m=+728.152950246" watchObservedRunningTime="2026-07-22 18:52:07.264293071 +0000 UTC m=+728.154834003"
Jul 22 18:52:07.278323 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.278298    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"]
Jul 22 18:52:07.282710 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.282688    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8jkmc"]
Jul 22 18:52:07.687863 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:07.687833    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1233c0d7-3813-4f5e-9352-dfef0cd6dc02" path="/var/lib/kubelet/pods/1233c0d7-3813-4f5e-9352-dfef0cd6dc02/volumes"
Jul 22 18:52:18.248906 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:18.248872    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 18:52:18.301242 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:18.301215    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"]
Jul 22 18:52:18.301431 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:18.301390    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l" podUID="8f3edca8-b7c7-4444-89c2-46dd0dc163ae" containerName="manager" containerID="cri-o://97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb" gracePeriod=10
Jul 22 18:52:19.122231 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.122204    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:52:19.209993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.209919    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-extensions-socket-volume\") pod \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\" (UID: \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\") "
Jul 22 18:52:19.209993 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.209993    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kn6cd\" (UniqueName: \"kubernetes.io/projected/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-kube-api-access-kn6cd\") pod \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\" (UID: \"8f3edca8-b7c7-4444-89c2-46dd0dc163ae\") "
Jul 22 18:52:19.210281 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.210256    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8f3edca8-b7c7-4444-89c2-46dd0dc163ae" (UID: "8f3edca8-b7c7-4444-89c2-46dd0dc163ae"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 18:52:19.211997 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.211977    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-kube-api-access-kn6cd" (OuterVolumeSpecName: "kube-api-access-kn6cd") pod "8f3edca8-b7c7-4444-89c2-46dd0dc163ae" (UID: "8f3edca8-b7c7-4444-89c2-46dd0dc163ae"). InnerVolumeSpecName "kube-api-access-kn6cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:52:19.293419 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.293390    2576 generic.go:356] "Generic (PLEG): container finished" podID="8f3edca8-b7c7-4444-89c2-46dd0dc163ae" containerID="97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb" exitCode=0
Jul 22 18:52:19.293800 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.293434    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"
Jul 22 18:52:19.293800 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.293498    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l" event={"ID":"8f3edca8-b7c7-4444-89c2-46dd0dc163ae","Type":"ContainerDied","Data":"97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb"}
Jul 22 18:52:19.293800 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.293549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l" event={"ID":"8f3edca8-b7c7-4444-89c2-46dd0dc163ae","Type":"ContainerDied","Data":"dc1537e9c35400f6df464e3c2fc5877d93f5e5a98a7115ecc166be0c10d6d9ab"}
Jul 22 18:52:19.293800 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.293572    2576 scope.go:117] "RemoveContainer" containerID="97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb"
Jul 22 18:52:19.302937 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.302765    2576 scope.go:117] "RemoveContainer" containerID="97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb"
Jul 22 18:52:19.303060 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:52:19.303027    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb\": container with ID starting with 97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb not found: ID does not exist" containerID="97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb"
Jul 22 18:52:19.303112 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.303054    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb"} err="failed to get container status \"97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb\": rpc error: code = NotFound desc = could not find container \"97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb\": container with ID starting with 97e015b3424798c565edecf2ab98ccc388b23f507c2e956de291b2af96ca83eb not found: ID does not exist"
Jul 22 18:52:19.310826 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.310809    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-extensions-socket-volume\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:52:19.310826 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.310827    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn6cd\" (UniqueName: \"kubernetes.io/projected/8f3edca8-b7c7-4444-89c2-46dd0dc163ae-kube-api-access-kn6cd\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:52:19.316116 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.316096    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"]
Jul 22 18:52:19.319514 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.319478    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cmb7l"]
Jul 22 18:52:19.687794 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:19.687763    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f3edca8-b7c7-4444-89c2-46dd0dc163ae" path="/var/lib/kubelet/pods/8f3edca8-b7c7-4444-89c2-46dd0dc163ae/volumes"
Jul 22 18:52:22.835500 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.835465    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"]
Jul 22 18:52:22.835885 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.835844    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f3edca8-b7c7-4444-89c2-46dd0dc163ae" containerName="manager"
Jul 22 18:52:22.835885 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.835858    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f3edca8-b7c7-4444-89c2-46dd0dc163ae" containerName="manager"
Jul 22 18:52:22.835954 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.835916    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f3edca8-b7c7-4444-89c2-46dd0dc163ae" containerName="manager"
Jul 22 18:52:22.872412 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.872380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"]
Jul 22 18:52:22.872564 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.872497    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:22.875158 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.875137    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8jwls\""
Jul 22 18:52:22.936590 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.936549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h2jp\" (UniqueName: \"kubernetes.io/projected/f6891e84-95bd-4474-bc17-43fd7b17f2c9-kube-api-access-5h2jp\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:22.936814 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.936608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:22.936814 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.936659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:22.936814 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.936722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:22.936814 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.936750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:22.936814 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.936806    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:22.937037 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.936832    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:22.937037 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.936924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:22.937037 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:22.936989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.037600 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.037568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.037600 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.037604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.037782 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.037649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.037782 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.037688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.037782 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.037716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h2jp\" (UniqueName: \"kubernetes.io/projected/f6891e84-95bd-4474-bc17-43fd7b17f2c9-kube-api-access-5h2jp\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.037782 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.037746    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.038010 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.037981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.038089 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.038006    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.038089 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.038081    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.038195 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.038122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.038195 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.038154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.038331 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.038215    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.038469 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.038449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.038682 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.038664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.039973 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.039952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.040184 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.040168    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.046227 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.046195    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6891e84-95bd-4474-bc17-43fd7b17f2c9-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.046320 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.046301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h2jp\" (UniqueName: \"kubernetes.io/projected/f6891e84-95bd-4474-bc17-43fd7b17f2c9-kube-api-access-5h2jp\") pod \"maas-default-gateway-openshift-default-56654dcf7-jrkgs\" (UID: \"f6891e84-95bd-4474-bc17-43fd7b17f2c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.182569 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.182542    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:23.313317 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.313289    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"]
Jul 22 18:52:23.315345 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:52:23.315312    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6891e84_95bd_4474_bc17_43fd7b17f2c9.slice/crio-2fc827ac5122685a29e61e4309de7570cd47bfade820d315c18e4a0ee35c4767 WatchSource:0}: Error finding container 2fc827ac5122685a29e61e4309de7570cd47bfade820d315c18e4a0ee35c4767: Status 404 returned error can't find the container with id 2fc827ac5122685a29e61e4309de7570cd47bfade820d315c18e4a0ee35c4767
Jul 22 18:52:23.317348 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.317318    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:52:23.317437 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.317396    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:52:23.317501 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:23.317437    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:52:24.318574 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:24.318538    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs" event={"ID":"f6891e84-95bd-4474-bc17-43fd7b17f2c9","Type":"ContainerStarted","Data":"e9d07e899028c0c7ebd063ec52596809d71aaa454aee9d0c6c51c5e8c398609e"}
Jul 22 18:52:24.319056 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:24.318599    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs" event={"ID":"f6891e84-95bd-4474-bc17-43fd7b17f2c9","Type":"ContainerStarted","Data":"2fc827ac5122685a29e61e4309de7570cd47bfade820d315c18e4a0ee35c4767"}
Jul 22 18:52:24.340905 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:24.340860    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs" podStartSLOduration=2.340844169 podStartE2EDuration="2.340844169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:52:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:24.33833248 +0000 UTC m=+745.228873424" watchObservedRunningTime="2026-07-22 18:52:24.340844169 +0000 UTC m=+745.231385101"
Jul 22 18:52:25.183622 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:25.183588    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:25.188708 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:25.188681    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:25.324117 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:25.324090    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:25.325075 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:25.325055    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:52:35.537853 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.537818    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-txnlj"]
Jul 22 18:52:35.572960 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.572930    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-txnlj"]
Jul 22 18:52:35.573114 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.573018    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:35.575537 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.575518    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 22 18:52:35.625195 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.625149    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-txnlj"]
Jul 22 18:52:35.742018 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.741985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7plw\" (UniqueName: \"kubernetes.io/projected/0edc6b22-4314-4548-8394-e6dfd6707d5e-kube-api-access-h7plw\") pod \"limitador-limitador-64c94fd64f-txnlj\" (UID: \"0edc6b22-4314-4548-8394-e6dfd6707d5e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:35.742018 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.742023    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0edc6b22-4314-4548-8394-e6dfd6707d5e-config-file\") pod \"limitador-limitador-64c94fd64f-txnlj\" (UID: \"0edc6b22-4314-4548-8394-e6dfd6707d5e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:35.842659 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.842556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7plw\" (UniqueName: \"kubernetes.io/projected/0edc6b22-4314-4548-8394-e6dfd6707d5e-kube-api-access-h7plw\") pod \"limitador-limitador-64c94fd64f-txnlj\" (UID: \"0edc6b22-4314-4548-8394-e6dfd6707d5e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:35.842659 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.842588    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0edc6b22-4314-4548-8394-e6dfd6707d5e-config-file\") pod \"limitador-limitador-64c94fd64f-txnlj\" (UID: \"0edc6b22-4314-4548-8394-e6dfd6707d5e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:35.843254 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.843233    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0edc6b22-4314-4548-8394-e6dfd6707d5e-config-file\") pod \"limitador-limitador-64c94fd64f-txnlj\" (UID: \"0edc6b22-4314-4548-8394-e6dfd6707d5e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:35.851344 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.851324    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7plw\" (UniqueName: \"kubernetes.io/projected/0edc6b22-4314-4548-8394-e6dfd6707d5e-kube-api-access-h7plw\") pod \"limitador-limitador-64c94fd64f-txnlj\" (UID: \"0edc6b22-4314-4548-8394-e6dfd6707d5e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:35.883071 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:35.883047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:36.013960 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.013932    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-txnlj"]
Jul 22 18:52:36.016284 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:52:36.016256    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0edc6b22_4314_4548_8394_e6dfd6707d5e.slice/crio-5ef4a18ca46ac461f739fda7fde3dfe30d0a77a3042daae051d2db7d4ed391f5 WatchSource:0}: Error finding container 5ef4a18ca46ac461f739fda7fde3dfe30d0a77a3042daae051d2db7d4ed391f5: Status 404 returned error can't find the container with id 5ef4a18ca46ac461f739fda7fde3dfe30d0a77a3042daae051d2db7d4ed391f5
Jul 22 18:52:36.348516 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.348474    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hxbns"]
Jul 22 18:52:36.352458 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.352436    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hxbns"
Jul 22 18:52:36.355241 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.355156    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vt7cj\""
Jul 22 18:52:36.358481 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.358457    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hxbns"]
Jul 22 18:52:36.378024 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.377994    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj" event={"ID":"0edc6b22-4314-4548-8394-e6dfd6707d5e","Type":"ContainerStarted","Data":"5ef4a18ca46ac461f739fda7fde3dfe30d0a77a3042daae051d2db7d4ed391f5"}
Jul 22 18:52:36.447850 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.447820    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74rwq\" (UniqueName: \"kubernetes.io/projected/19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b-kube-api-access-74rwq\") pod \"authorino-58cd955675-hxbns\" (UID: \"19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b\") " pod="kuadrant-system/authorino-58cd955675-hxbns"
Jul 22 18:52:36.530152 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.530124    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgltj"]
Jul 22 18:52:36.532980 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.532962    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bgltj"
Jul 22 18:52:36.540081 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.540061    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgltj"]
Jul 22 18:52:36.548754 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.548716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74rwq\" (UniqueName: \"kubernetes.io/projected/19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b-kube-api-access-74rwq\") pod \"authorino-58cd955675-hxbns\" (UID: \"19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b\") " pod="kuadrant-system/authorino-58cd955675-hxbns"
Jul 22 18:52:36.556817 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.556792    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74rwq\" (UniqueName: \"kubernetes.io/projected/19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b-kube-api-access-74rwq\") pod \"authorino-58cd955675-hxbns\" (UID: \"19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b\") " pod="kuadrant-system/authorino-58cd955675-hxbns"
Jul 22 18:52:36.649078 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.649055    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88sm5\" (UniqueName: \"kubernetes.io/projected/4c159543-04f0-44f6-83e2-dae6a6861039-kube-api-access-88sm5\") pod \"authorino-75d556bd77-bgltj\" (UID: \"4c159543-04f0-44f6-83e2-dae6a6861039\") " pod="kuadrant-system/authorino-75d556bd77-bgltj"
Jul 22 18:52:36.662476 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.662456    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hxbns"
Jul 22 18:52:36.750234 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.750205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88sm5\" (UniqueName: \"kubernetes.io/projected/4c159543-04f0-44f6-83e2-dae6a6861039-kube-api-access-88sm5\") pod \"authorino-75d556bd77-bgltj\" (UID: \"4c159543-04f0-44f6-83e2-dae6a6861039\") " pod="kuadrant-system/authorino-75d556bd77-bgltj"
Jul 22 18:52:36.760834 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.760804    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88sm5\" (UniqueName: \"kubernetes.io/projected/4c159543-04f0-44f6-83e2-dae6a6861039-kube-api-access-88sm5\") pod \"authorino-75d556bd77-bgltj\" (UID: \"4c159543-04f0-44f6-83e2-dae6a6861039\") " pod="kuadrant-system/authorino-75d556bd77-bgltj"
Jul 22 18:52:36.783963 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.783941    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hxbns"]
Jul 22 18:52:36.786220 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:52:36.786196    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19f3ebeb_77c5_48cf_b1cc_fe3a2bf8714b.slice/crio-a00b75872d41e1cd62e934641b856eda0bf7894971c178846a68c9b16515c9d1 WatchSource:0}: Error finding container a00b75872d41e1cd62e934641b856eda0bf7894971c178846a68c9b16515c9d1: Status 404 returned error can't find the container with id a00b75872d41e1cd62e934641b856eda0bf7894971c178846a68c9b16515c9d1
Jul 22 18:52:36.842980 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.842953    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bgltj"
Jul 22 18:52:36.968154 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:36.968124    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgltj"]
Jul 22 18:52:36.970878 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:52:36.970852    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c159543_04f0_44f6_83e2_dae6a6861039.slice/crio-044fb47c6c058a2ba049245bfc8799468d6ac1955beea6fcaab6a7a64fd7f586 WatchSource:0}: Error finding container 044fb47c6c058a2ba049245bfc8799468d6ac1955beea6fcaab6a7a64fd7f586: Status 404 returned error can't find the container with id 044fb47c6c058a2ba049245bfc8799468d6ac1955beea6fcaab6a7a64fd7f586
Jul 22 18:52:37.385371 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:37.385338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hxbns" event={"ID":"19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b","Type":"ContainerStarted","Data":"a00b75872d41e1cd62e934641b856eda0bf7894971c178846a68c9b16515c9d1"}
Jul 22 18:52:37.386916 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:37.386892    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bgltj" event={"ID":"4c159543-04f0-44f6-83e2-dae6a6861039","Type":"ContainerStarted","Data":"044fb47c6c058a2ba049245bfc8799468d6ac1955beea6fcaab6a7a64fd7f586"}
Jul 22 18:52:42.418483 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:42.418438    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hxbns" event={"ID":"19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b","Type":"ContainerStarted","Data":"4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef"}
Jul 22 18:52:42.419958 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:42.419928    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj" event={"ID":"0edc6b22-4314-4548-8394-e6dfd6707d5e","Type":"ContainerStarted","Data":"a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792"}
Jul 22 18:52:42.420069 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:42.420027    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:42.421395 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:42.421370    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bgltj" event={"ID":"4c159543-04f0-44f6-83e2-dae6a6861039","Type":"ContainerStarted","Data":"f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd"}
Jul 22 18:52:42.432513 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:42.432471    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hxbns" podStartSLOduration=1.6943132969999999 podStartE2EDuration="6.432459005s" totalImagesPullingTime="4.738145708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:52:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:42.432040603 +0000 UTC m=+763.322581534" watchObservedRunningTime="2026-07-22 18:52:42.432459005 +0000 UTC m=+763.322999936"
Jul 22 18:52:42.449354 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:42.449309    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj" podStartSLOduration=1.892464995 podStartE2EDuration="7.449295984s" totalImagesPullingTime="5.556830989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:42.447839485 +0000 UTC m=+763.338380416" watchObservedRunningTime="2026-07-22 18:52:42.449295984 +0000 UTC m=+763.339836915"
Jul 22 18:52:42.461477 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:42.461430    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-bgltj" podStartSLOduration=1.873033792 podStartE2EDuration="6.46141837s" totalImagesPullingTime="4.588384578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:52:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:42.460771283 +0000 UTC m=+763.351312216" watchObservedRunningTime="2026-07-22 18:52:42.46141837 +0000 UTC m=+763.351959302"
Jul 22 18:52:42.486213 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:42.486185    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hxbns"]
Jul 22 18:52:44.430050 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:44.430011    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hxbns" podUID="19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b" containerName="authorino" containerID="cri-o://4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef" gracePeriod=30
Jul 22 18:52:44.671001 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:44.670978    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hxbns"
Jul 22 18:52:44.824729 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:44.824654    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74rwq\" (UniqueName: \"kubernetes.io/projected/19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b-kube-api-access-74rwq\") pod \"19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b\" (UID: \"19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b\") "
Jul 22 18:52:44.826576 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:44.826543    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b-kube-api-access-74rwq" (OuterVolumeSpecName: "kube-api-access-74rwq") pod "19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b" (UID: "19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b"). InnerVolumeSpecName "kube-api-access-74rwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:52:44.926128 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:44.926102    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74rwq\" (UniqueName: \"kubernetes.io/projected/19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b-kube-api-access-74rwq\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:52:45.436703 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.436670    2576 generic.go:356] "Generic (PLEG): container finished" podID="19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b" containerID="4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef" exitCode=0
Jul 22 18:52:45.437146 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.436712    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hxbns"
Jul 22 18:52:45.437146 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.436762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hxbns" event={"ID":"19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b","Type":"ContainerDied","Data":"4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef"}
Jul 22 18:52:45.437146 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.436799    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hxbns" event={"ID":"19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b","Type":"ContainerDied","Data":"a00b75872d41e1cd62e934641b856eda0bf7894971c178846a68c9b16515c9d1"}
Jul 22 18:52:45.437146 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.436817    2576 scope.go:117] "RemoveContainer" containerID="4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef"
Jul 22 18:52:45.447831 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.447815    2576 scope.go:117] "RemoveContainer" containerID="4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef"
Jul 22 18:52:45.448101 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:52:45.448080    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef\": container with ID starting with 4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef not found: ID does not exist" containerID="4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef"
Jul 22 18:52:45.448214 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.448105    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef"} err="failed to get container status \"4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef\": rpc error: code = NotFound desc = could not find container \"4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef\": container with ID starting with 4464da975be50442bd0ee2b4718fca21952b68e953a2daa9b5837a0158a802ef not found: ID does not exist"
Jul 22 18:52:45.460155 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.460133    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hxbns"]
Jul 22 18:52:45.464115 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.464095    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hxbns"]
Jul 22 18:52:45.688719 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:45.688651    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b" path="/var/lib/kubelet/pods/19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b/volumes"
Jul 22 18:52:50.389147 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:50.389116    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-txnlj"]
Jul 22 18:52:50.389678 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:50.389314    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj" podUID="0edc6b22-4314-4548-8394-e6dfd6707d5e" containerName="limitador" containerID="cri-o://a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792" gracePeriod=30
Jul 22 18:52:50.391494 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:50.391468    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:50.940138 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:50.940115    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:51.073983 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.073905    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7plw\" (UniqueName: \"kubernetes.io/projected/0edc6b22-4314-4548-8394-e6dfd6707d5e-kube-api-access-h7plw\") pod \"0edc6b22-4314-4548-8394-e6dfd6707d5e\" (UID: \"0edc6b22-4314-4548-8394-e6dfd6707d5e\") "
Jul 22 18:52:51.073983 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.073964    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0edc6b22-4314-4548-8394-e6dfd6707d5e-config-file\") pod \"0edc6b22-4314-4548-8394-e6dfd6707d5e\" (UID: \"0edc6b22-4314-4548-8394-e6dfd6707d5e\") "
Jul 22 18:52:51.074329 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.074304    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0edc6b22-4314-4548-8394-e6dfd6707d5e-config-file" (OuterVolumeSpecName: "config-file") pod "0edc6b22-4314-4548-8394-e6dfd6707d5e" (UID: "0edc6b22-4314-4548-8394-e6dfd6707d5e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 18:52:51.075992 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.075961    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0edc6b22-4314-4548-8394-e6dfd6707d5e-kube-api-access-h7plw" (OuterVolumeSpecName: "kube-api-access-h7plw") pod "0edc6b22-4314-4548-8394-e6dfd6707d5e" (UID: "0edc6b22-4314-4548-8394-e6dfd6707d5e"). InnerVolumeSpecName "kube-api-access-h7plw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:52:51.174484 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.174455    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7plw\" (UniqueName: \"kubernetes.io/projected/0edc6b22-4314-4548-8394-e6dfd6707d5e-kube-api-access-h7plw\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:52:51.174484 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.174480    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0edc6b22-4314-4548-8394-e6dfd6707d5e-config-file\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:52:51.463073 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.463043    2576 generic.go:356] "Generic (PLEG): container finished" podID="0edc6b22-4314-4548-8394-e6dfd6707d5e" containerID="a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792" exitCode=0
Jul 22 18:52:51.463474 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.463086    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj"
Jul 22 18:52:51.463474 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.463107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj" event={"ID":"0edc6b22-4314-4548-8394-e6dfd6707d5e","Type":"ContainerDied","Data":"a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792"}
Jul 22 18:52:51.463474 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.463145    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-txnlj" event={"ID":"0edc6b22-4314-4548-8394-e6dfd6707d5e","Type":"ContainerDied","Data":"5ef4a18ca46ac461f739fda7fde3dfe30d0a77a3042daae051d2db7d4ed391f5"}
Jul 22 18:52:51.463474 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.463166    2576 scope.go:117] "RemoveContainer" containerID="a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792"
Jul 22 18:52:51.471357 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.471341    2576 scope.go:117] "RemoveContainer" containerID="a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792"
Jul 22 18:52:51.471591 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:52:51.471576    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792\": container with ID starting with a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792 not found: ID does not exist" containerID="a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792"
Jul 22 18:52:51.471645 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.471599    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792"} err="failed to get container status \"a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792\": rpc error: code = NotFound desc = could not find container \"a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792\": container with ID starting with a4e756d1c3a0117445deb6deace7d56ff4039180624a9a963993820c980a3792 not found: ID does not exist"
Jul 22 18:52:51.485452 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.485431    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-txnlj"]
Jul 22 18:52:51.489399 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.489377    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-txnlj"]
Jul 22 18:52:51.687692 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:51.687663    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0edc6b22-4314-4548-8394-e6dfd6707d5e" path="/var/lib/kubelet/pods/0edc6b22-4314-4548-8394-e6dfd6707d5e/volumes"
Jul 22 18:52:52.429550 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.429520    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5x299"]
Jul 22 18:52:52.429985 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.429967    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0edc6b22-4314-4548-8394-e6dfd6707d5e" containerName="limitador"
Jul 22 18:52:52.429985 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.429986    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0edc6b22-4314-4548-8394-e6dfd6707d5e" containerName="limitador"
Jul 22 18:52:52.430154 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.430030    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b" containerName="authorino"
Jul 22 18:52:52.430154 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.430040    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b" containerName="authorino"
Jul 22 18:52:52.430295 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.430272    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0edc6b22-4314-4548-8394-e6dfd6707d5e" containerName="limitador"
Jul 22 18:52:52.430295 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.430292    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="19f3ebeb-77c5-48cf-b1cc-fe3a2bf8714b" containerName="authorino"
Jul 22 18:52:52.434559 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.434542    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:52:52.437515 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.437493    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 18:52:52.437621 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.437493    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 18:52:52.438444 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.438429    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 22 18:52:52.438520 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.438461    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4cchf\""
Jul 22 18:52:52.445290 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.445270    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5x299"]
Jul 22 18:52:52.484553 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.484532    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/43abc0ed-8a8d-46cd-9c29-1e0dfac0f274-data\") pod \"postgres-69fb4594cf-5x299\" (UID: \"43abc0ed-8a8d-46cd-9c29-1e0dfac0f274\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:52:52.484873 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.484581    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2gr2\" (UniqueName: \"kubernetes.io/projected/43abc0ed-8a8d-46cd-9c29-1e0dfac0f274-kube-api-access-f2gr2\") pod \"postgres-69fb4594cf-5x299\" (UID: \"43abc0ed-8a8d-46cd-9c29-1e0dfac0f274\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:52:52.585405 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.585380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/43abc0ed-8a8d-46cd-9c29-1e0dfac0f274-data\") pod \"postgres-69fb4594cf-5x299\" (UID: \"43abc0ed-8a8d-46cd-9c29-1e0dfac0f274\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:52:52.585576 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.585428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2gr2\" (UniqueName: \"kubernetes.io/projected/43abc0ed-8a8d-46cd-9c29-1e0dfac0f274-kube-api-access-f2gr2\") pod \"postgres-69fb4594cf-5x299\" (UID: \"43abc0ed-8a8d-46cd-9c29-1e0dfac0f274\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:52:52.585834 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.585817    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/43abc0ed-8a8d-46cd-9c29-1e0dfac0f274-data\") pod \"postgres-69fb4594cf-5x299\" (UID: \"43abc0ed-8a8d-46cd-9c29-1e0dfac0f274\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:52:52.596015 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.595986    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2gr2\" (UniqueName: \"kubernetes.io/projected/43abc0ed-8a8d-46cd-9c29-1e0dfac0f274-kube-api-access-f2gr2\") pod \"postgres-69fb4594cf-5x299\" (UID: \"43abc0ed-8a8d-46cd-9c29-1e0dfac0f274\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:52:52.744469 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.744404    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:52:52.874759 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:52.874735    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5x299"]
Jul 22 18:52:52.876919 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:52:52.876890    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43abc0ed_8a8d_46cd_9c29_1e0dfac0f274.slice/crio-1ce6ac35be4c0f1b316143b2c8455e0ee019f38057d45bc0eb0de8c489e9d458 WatchSource:0}: Error finding container 1ce6ac35be4c0f1b316143b2c8455e0ee019f38057d45bc0eb0de8c489e9d458: Status 404 returned error can't find the container with id 1ce6ac35be4c0f1b316143b2c8455e0ee019f38057d45bc0eb0de8c489e9d458
Jul 22 18:52:53.474193 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:53.474158    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299" event={"ID":"43abc0ed-8a8d-46cd-9c29-1e0dfac0f274","Type":"ContainerStarted","Data":"1ce6ac35be4c0f1b316143b2c8455e0ee019f38057d45bc0eb0de8c489e9d458"}
Jul 22 18:52:58.504568 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:58.504537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299" event={"ID":"43abc0ed-8a8d-46cd-9c29-1e0dfac0f274","Type":"ContainerStarted","Data":"8e598c0482725f4e12a217140196240a20d8a19c43ca498785e5a9f478fc8523"}
Jul 22 18:52:58.504985 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:58.504685    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:52:58.521216 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:52:58.521169    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299" podStartSLOduration=0.995876767 podStartE2EDuration="6.5211554s" totalImagesPullingTime="5.525278633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:58.519969845 +0000 UTC m=+779.410510777" watchObservedRunningTime="2026-07-22 18:52:58.5211554 +0000 UTC m=+779.411696332"
Jul 22 18:53:04.538872 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:04.538846    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x299"
Jul 22 18:53:05.342260 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.342225    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-rx9n9"]
Jul 22 18:53:05.347996 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.347979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:05.350739 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.350717    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 22 18:53:05.358876 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.358852    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-rx9n9"]
Jul 22 18:53:05.382372 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.382348    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-rx9n9"]
Jul 22 18:53:05.382558 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:53:05.382542    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-76gjb tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-rx9n9" podUID="7cf6e37a-f2c2-452f-9531-9b43e30e6be3"
Jul 22 18:53:05.387094 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.387071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76gjb\" (UniqueName: \"kubernetes.io/projected/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-kube-api-access-76gjb\") pod \"authorino-76877994-rx9n9\" (UID: \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\") " pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:05.387169 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.387129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-tls-cert\") pod \"authorino-76877994-rx9n9\" (UID: \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\") " pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:05.424068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.424049    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67cf47bbfd-sm9lg"]
Jul 22 18:53:05.427507 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.427492    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:53:05.435359 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.435339    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67cf47bbfd-sm9lg"]
Jul 22 18:53:05.487539 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.487505    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djvrw\" (UniqueName: \"kubernetes.io/projected/eb996e96-8754-4ef6-90b3-e41c4604319a-kube-api-access-djvrw\") pod \"authorino-67cf47bbfd-sm9lg\" (UID: \"eb996e96-8754-4ef6-90b3-e41c4604319a\") " pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:53:05.487539 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.487537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/eb996e96-8754-4ef6-90b3-e41c4604319a-tls-cert\") pod \"authorino-67cf47bbfd-sm9lg\" (UID: \"eb996e96-8754-4ef6-90b3-e41c4604319a\") " pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:53:05.487734 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.487561    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-tls-cert\") pod \"authorino-76877994-rx9n9\" (UID: \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\") " pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:05.487734 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.487672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76gjb\" (UniqueName: \"kubernetes.io/projected/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-kube-api-access-76gjb\") pod \"authorino-76877994-rx9n9\" (UID: \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\") " pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:05.489903 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.489884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-tls-cert\") pod \"authorino-76877994-rx9n9\" (UID: \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\") " pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:05.495999 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.495979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76gjb\" (UniqueName: \"kubernetes.io/projected/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-kube-api-access-76gjb\") pod \"authorino-76877994-rx9n9\" (UID: \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\") " pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:05.537238 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.537218    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:05.544167 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.544148    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:05.588706 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.588687    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76gjb\" (UniqueName: \"kubernetes.io/projected/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-kube-api-access-76gjb\") pod \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\" (UID: \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\") "
Jul 22 18:53:05.588816 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.588754    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-tls-cert\") pod \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\" (UID: \"7cf6e37a-f2c2-452f-9531-9b43e30e6be3\") "
Jul 22 18:53:05.588928 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.588912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djvrw\" (UniqueName: \"kubernetes.io/projected/eb996e96-8754-4ef6-90b3-e41c4604319a-kube-api-access-djvrw\") pod \"authorino-67cf47bbfd-sm9lg\" (UID: \"eb996e96-8754-4ef6-90b3-e41c4604319a\") " pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:53:05.588987 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.588944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/eb996e96-8754-4ef6-90b3-e41c4604319a-tls-cert\") pod \"authorino-67cf47bbfd-sm9lg\" (UID: \"eb996e96-8754-4ef6-90b3-e41c4604319a\") " pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:53:05.590598 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.590571    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-kube-api-access-76gjb" (OuterVolumeSpecName: "kube-api-access-76gjb") pod "7cf6e37a-f2c2-452f-9531-9b43e30e6be3" (UID: "7cf6e37a-f2c2-452f-9531-9b43e30e6be3"). InnerVolumeSpecName "kube-api-access-76gjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:53:05.590697 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.590677    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7cf6e37a-f2c2-452f-9531-9b43e30e6be3" (UID: "7cf6e37a-f2c2-452f-9531-9b43e30e6be3"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:53:05.591145 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.591130    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/eb996e96-8754-4ef6-90b3-e41c4604319a-tls-cert\") pod \"authorino-67cf47bbfd-sm9lg\" (UID: \"eb996e96-8754-4ef6-90b3-e41c4604319a\") " pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:53:05.597692 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.597671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djvrw\" (UniqueName: \"kubernetes.io/projected/eb996e96-8754-4ef6-90b3-e41c4604319a-kube-api-access-djvrw\") pod \"authorino-67cf47bbfd-sm9lg\" (UID: \"eb996e96-8754-4ef6-90b3-e41c4604319a\") " pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:53:05.689756 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.689730    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-tls-cert\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:53:05.689756 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.689754    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76gjb\" (UniqueName: \"kubernetes.io/projected/7cf6e37a-f2c2-452f-9531-9b43e30e6be3-kube-api-access-76gjb\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:53:05.736718 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.736697    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:53:05.862665 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:05.862622    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67cf47bbfd-sm9lg"]
Jul 22 18:53:05.864759 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:53:05.864732    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb996e96_8754_4ef6_90b3_e41c4604319a.slice/crio-168fd0decdbe0759f149147af4fcb7b81df233312590b1c979039e488304d11e WatchSource:0}: Error finding container 168fd0decdbe0759f149147af4fcb7b81df233312590b1c979039e488304d11e: Status 404 returned error can't find the container with id 168fd0decdbe0759f149147af4fcb7b81df233312590b1c979039e488304d11e
Jul 22 18:53:06.543693 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.543593    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-rx9n9"
Jul 22 18:53:06.543693 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.543588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg" event={"ID":"eb996e96-8754-4ef6-90b3-e41c4604319a","Type":"ContainerStarted","Data":"ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211"}
Jul 22 18:53:06.543693 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.543673    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg" event={"ID":"eb996e96-8754-4ef6-90b3-e41c4604319a","Type":"ContainerStarted","Data":"168fd0decdbe0759f149147af4fcb7b81df233312590b1c979039e488304d11e"}
Jul 22 18:53:06.567640 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.567591    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg" podStartSLOduration=1.260162759 podStartE2EDuration="1.567578823s" totalImagesPullingTime="307.416064ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:53:06.567298827 +0000 UTC m=+787.457839759" watchObservedRunningTime="2026-07-22 18:53:06.567578823 +0000 UTC m=+787.458119787"
Jul 22 18:53:06.595306 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.595279    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-rx9n9"]
Jul 22 18:53:06.601605 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.601579    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgltj"]
Jul 22 18:53:06.601846 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.601819    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-bgltj" podUID="4c159543-04f0-44f6-83e2-dae6a6861039" containerName="authorino" containerID="cri-o://f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd" gracePeriod=30
Jul 22 18:53:06.603052 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.603031    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-rx9n9"]
Jul 22 18:53:06.853258 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.853234    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bgltj"
Jul 22 18:53:06.897885 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.897854    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88sm5\" (UniqueName: \"kubernetes.io/projected/4c159543-04f0-44f6-83e2-dae6a6861039-kube-api-access-88sm5\") pod \"4c159543-04f0-44f6-83e2-dae6a6861039\" (UID: \"4c159543-04f0-44f6-83e2-dae6a6861039\") "
Jul 22 18:53:06.899932 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.899903    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c159543-04f0-44f6-83e2-dae6a6861039-kube-api-access-88sm5" (OuterVolumeSpecName: "kube-api-access-88sm5") pod "4c159543-04f0-44f6-83e2-dae6a6861039" (UID: "4c159543-04f0-44f6-83e2-dae6a6861039"). InnerVolumeSpecName "kube-api-access-88sm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:53:06.998783 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:06.998746    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88sm5\" (UniqueName: \"kubernetes.io/projected/4c159543-04f0-44f6-83e2-dae6a6861039-kube-api-access-88sm5\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:53:07.549609 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.549574    2576 generic.go:356] "Generic (PLEG): container finished" podID="4c159543-04f0-44f6-83e2-dae6a6861039" containerID="f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd" exitCode=0
Jul 22 18:53:07.549798 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.549614    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bgltj"
Jul 22 18:53:07.549798 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.549669    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bgltj" event={"ID":"4c159543-04f0-44f6-83e2-dae6a6861039","Type":"ContainerDied","Data":"f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd"}
Jul 22 18:53:07.549798 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.549712    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bgltj" event={"ID":"4c159543-04f0-44f6-83e2-dae6a6861039","Type":"ContainerDied","Data":"044fb47c6c058a2ba049245bfc8799468d6ac1955beea6fcaab6a7a64fd7f586"}
Jul 22 18:53:07.549798 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.549729    2576 scope.go:117] "RemoveContainer" containerID="f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd"
Jul 22 18:53:07.557959 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.557944    2576 scope.go:117] "RemoveContainer" containerID="f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd"
Jul 22 18:53:07.558196 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:53:07.558179    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd\": container with ID starting with f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd not found: ID does not exist" containerID="f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd"
Jul 22 18:53:07.558237 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.558205    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd"} err="failed to get container status \"f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd\": rpc error: code = NotFound desc = could not find container \"f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd\": container with ID starting with f7b0cce6986df1c80207fa566c3bb30d58e4e3eb327fb366c96bb493ea8b78cd not found: ID does not exist"
Jul 22 18:53:07.571515 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.571483    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgltj"]
Jul 22 18:53:07.576009 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.575989    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgltj"]
Jul 22 18:53:07.688737 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.688714    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c159543-04f0-44f6-83e2-dae6a6861039" path="/var/lib/kubelet/pods/4c159543-04f0-44f6-83e2-dae6a6861039/volumes"
Jul 22 18:53:07.689064 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:07.689052    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cf6e37a-f2c2-452f-9531-9b43e30e6be3" path="/var/lib/kubelet/pods/7cf6e37a-f2c2-452f-9531-9b43e30e6be3/volumes"
Jul 22 18:53:11.456020 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.455985    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75d9999fb9-2f6z5"]
Jul 22 18:53:11.456452 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.456293    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c159543-04f0-44f6-83e2-dae6a6861039" containerName="authorino"
Jul 22 18:53:11.456452 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.456306    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c159543-04f0-44f6-83e2-dae6a6861039" containerName="authorino"
Jul 22 18:53:11.456452 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.456369    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c159543-04f0-44f6-83e2-dae6a6861039" containerName="authorino"
Jul 22 18:53:11.605566 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.605533    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75d9999fb9-2f6z5"]
Jul 22 18:53:11.605741 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.605653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:11.609496 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.609466    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 18:53:11.609660 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.609507    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 18:53:11.609660 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.609466    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rv8cf\""
Jul 22 18:53:11.642485 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.642464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cdf4ef4-3348-47df-8922-d4704a175874-webhook-cert\") pod \"maas-controller-75d9999fb9-2f6z5\" (UID: \"6cdf4ef4-3348-47df-8922-d4704a175874\") " pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:11.642616 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.642526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cnc9\" (UniqueName: \"kubernetes.io/projected/6cdf4ef4-3348-47df-8922-d4704a175874-kube-api-access-8cnc9\") pod \"maas-controller-75d9999fb9-2f6z5\" (UID: \"6cdf4ef4-3348-47df-8922-d4704a175874\") " pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:11.643047 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.643026    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9d64f8564-wqbqd"]
Jul 22 18:53:11.673334 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.673305    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9d64f8564-wqbqd"]
Jul 22 18:53:11.673448 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.673413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:11.743901 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.743837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cnc9\" (UniqueName: \"kubernetes.io/projected/6cdf4ef4-3348-47df-8922-d4704a175874-kube-api-access-8cnc9\") pod \"maas-controller-75d9999fb9-2f6z5\" (UID: \"6cdf4ef4-3348-47df-8922-d4704a175874\") " pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:11.743901 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.743873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmttj\" (UniqueName: \"kubernetes.io/projected/36f7ae02-7fbe-437c-be3d-b140c8477a48-kube-api-access-dmttj\") pod \"maas-controller-9d64f8564-wqbqd\" (UID: \"36f7ae02-7fbe-437c-be3d-b140c8477a48\") " pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:11.743901 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.743896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cdf4ef4-3348-47df-8922-d4704a175874-webhook-cert\") pod \"maas-controller-75d9999fb9-2f6z5\" (UID: \"6cdf4ef4-3348-47df-8922-d4704a175874\") " pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:11.744077 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.743974    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36f7ae02-7fbe-437c-be3d-b140c8477a48-webhook-cert\") pod \"maas-controller-9d64f8564-wqbqd\" (UID: \"36f7ae02-7fbe-437c-be3d-b140c8477a48\") " pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:11.746151 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.746133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cdf4ef4-3348-47df-8922-d4704a175874-webhook-cert\") pod \"maas-controller-75d9999fb9-2f6z5\" (UID: \"6cdf4ef4-3348-47df-8922-d4704a175874\") " pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:11.752688 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.752666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cnc9\" (UniqueName: \"kubernetes.io/projected/6cdf4ef4-3348-47df-8922-d4704a175874-kube-api-access-8cnc9\") pod \"maas-controller-75d9999fb9-2f6z5\" (UID: \"6cdf4ef4-3348-47df-8922-d4704a175874\") " pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:11.844487 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.844458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36f7ae02-7fbe-437c-be3d-b140c8477a48-webhook-cert\") pod \"maas-controller-9d64f8564-wqbqd\" (UID: \"36f7ae02-7fbe-437c-be3d-b140c8477a48\") " pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:11.844609 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.844501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmttj\" (UniqueName: \"kubernetes.io/projected/36f7ae02-7fbe-437c-be3d-b140c8477a48-kube-api-access-dmttj\") pod \"maas-controller-9d64f8564-wqbqd\" (UID: \"36f7ae02-7fbe-437c-be3d-b140c8477a48\") " pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:11.846708 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.846682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36f7ae02-7fbe-437c-be3d-b140c8477a48-webhook-cert\") pod \"maas-controller-9d64f8564-wqbqd\" (UID: \"36f7ae02-7fbe-437c-be3d-b140c8477a48\") " pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:11.854962 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.854939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmttj\" (UniqueName: \"kubernetes.io/projected/36f7ae02-7fbe-437c-be3d-b140c8477a48-kube-api-access-dmttj\") pod \"maas-controller-9d64f8564-wqbqd\" (UID: \"36f7ae02-7fbe-437c-be3d-b140c8477a48\") " pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:11.915520 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.915495    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:11.982945 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:11.982911    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:12.082309 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:12.082281    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75d9999fb9-2f6z5"]
Jul 22 18:53:12.084429 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:53:12.084405    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cdf4ef4_3348_47df_8922_d4704a175874.slice/crio-cdca6b2afc8ef65ea7e01acf6297e55c3a27e131c5c4a1ca892814d1ed2b6287 WatchSource:0}: Error finding container cdca6b2afc8ef65ea7e01acf6297e55c3a27e131c5c4a1ca892814d1ed2b6287: Status 404 returned error can't find the container with id cdca6b2afc8ef65ea7e01acf6297e55c3a27e131c5c4a1ca892814d1ed2b6287
Jul 22 18:53:12.143869 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:12.143840    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9d64f8564-wqbqd"]
Jul 22 18:53:12.146584 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:53:12.146550    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f7ae02_7fbe_437c_be3d_b140c8477a48.slice/crio-43666ae40e26d5d0c2cdd21a1b7f11a58744ee3fd41ab3b30dc3cfa958d451a3 WatchSource:0}: Error finding container 43666ae40e26d5d0c2cdd21a1b7f11a58744ee3fd41ab3b30dc3cfa958d451a3: Status 404 returned error can't find the container with id 43666ae40e26d5d0c2cdd21a1b7f11a58744ee3fd41ab3b30dc3cfa958d451a3
Jul 22 18:53:12.573294 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:12.573259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75d9999fb9-2f6z5" event={"ID":"6cdf4ef4-3348-47df-8922-d4704a175874","Type":"ContainerStarted","Data":"cdca6b2afc8ef65ea7e01acf6297e55c3a27e131c5c4a1ca892814d1ed2b6287"}
Jul 22 18:53:12.574278 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:12.574252    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-wqbqd" event={"ID":"36f7ae02-7fbe-437c-be3d-b140c8477a48","Type":"ContainerStarted","Data":"43666ae40e26d5d0c2cdd21a1b7f11a58744ee3fd41ab3b30dc3cfa958d451a3"}
Jul 22 18:53:15.596093 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:15.596056    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75d9999fb9-2f6z5" event={"ID":"6cdf4ef4-3348-47df-8922-d4704a175874","Type":"ContainerStarted","Data":"a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c"}
Jul 22 18:53:15.596538 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:15.596163    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:15.597474 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:15.597452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-wqbqd" event={"ID":"36f7ae02-7fbe-437c-be3d-b140c8477a48","Type":"ContainerStarted","Data":"7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f"}
Jul 22 18:53:15.597551 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:15.597531    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:15.633884 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:15.633837    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75d9999fb9-2f6z5" podStartSLOduration=1.731871356 podStartE2EDuration="4.633823274s" totalImagesPullingTime="2.901951918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:53:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:53:15.631009394 +0000 UTC m=+796.521550327" watchObservedRunningTime="2026-07-22 18:53:15.633823274 +0000 UTC m=+796.524364206"
Jul 22 18:53:15.666336 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:15.666275    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9d64f8564-wqbqd" podStartSLOduration=1.832029821 podStartE2EDuration="4.666260276s" totalImagesPullingTime="2.834230455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:53:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:53:15.66542466 +0000 UTC m=+796.555965593" watchObservedRunningTime="2026-07-22 18:53:15.666260276 +0000 UTC m=+796.556801207"
Jul 22 18:53:26.604697 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:26.604576    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:26.604697 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:26.604691    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:53:26.682106 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:26.682067    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75d9999fb9-2f6z5"]
Jul 22 18:53:26.682435 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:26.682377    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75d9999fb9-2f6z5" podUID="6cdf4ef4-3348-47df-8922-d4704a175874" containerName="manager" containerID="cri-o://a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c" gracePeriod=30
Jul 22 18:53:26.928623 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:26.928602    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:27.065684 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.065648    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cnc9\" (UniqueName: \"kubernetes.io/projected/6cdf4ef4-3348-47df-8922-d4704a175874-kube-api-access-8cnc9\") pod \"6cdf4ef4-3348-47df-8922-d4704a175874\" (UID: \"6cdf4ef4-3348-47df-8922-d4704a175874\") "
Jul 22 18:53:27.065865 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.065721    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cdf4ef4-3348-47df-8922-d4704a175874-webhook-cert\") pod \"6cdf4ef4-3348-47df-8922-d4704a175874\" (UID: \"6cdf4ef4-3348-47df-8922-d4704a175874\") "
Jul 22 18:53:27.067854 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.067816    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cdf4ef4-3348-47df-8922-d4704a175874-kube-api-access-8cnc9" (OuterVolumeSpecName: "kube-api-access-8cnc9") pod "6cdf4ef4-3348-47df-8922-d4704a175874" (UID: "6cdf4ef4-3348-47df-8922-d4704a175874"). InnerVolumeSpecName "kube-api-access-8cnc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:53:27.067971 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.067864    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdf4ef4-3348-47df-8922-d4704a175874-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6cdf4ef4-3348-47df-8922-d4704a175874" (UID: "6cdf4ef4-3348-47df-8922-d4704a175874"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:53:27.167203 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.167121    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cdf4ef4-3348-47df-8922-d4704a175874-webhook-cert\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:53:27.167203 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.167149    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cnc9\" (UniqueName: \"kubernetes.io/projected/6cdf4ef4-3348-47df-8922-d4704a175874-kube-api-access-8cnc9\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:53:27.651735 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.651704    2576 generic.go:356] "Generic (PLEG): container finished" podID="6cdf4ef4-3348-47df-8922-d4704a175874" containerID="a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c" exitCode=0
Jul 22 18:53:27.652196 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.651758    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75d9999fb9-2f6z5"
Jul 22 18:53:27.652196 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.651764    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75d9999fb9-2f6z5" event={"ID":"6cdf4ef4-3348-47df-8922-d4704a175874","Type":"ContainerDied","Data":"a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c"}
Jul 22 18:53:27.652196 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.651827    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75d9999fb9-2f6z5" event={"ID":"6cdf4ef4-3348-47df-8922-d4704a175874","Type":"ContainerDied","Data":"cdca6b2afc8ef65ea7e01acf6297e55c3a27e131c5c4a1ca892814d1ed2b6287"}
Jul 22 18:53:27.652196 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.651860    2576 scope.go:117] "RemoveContainer" containerID="a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c"
Jul 22 18:53:27.660765 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.660752    2576 scope.go:117] "RemoveContainer" containerID="a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c"
Jul 22 18:53:27.661007 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:53:27.660989    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c\": container with ID starting with a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c not found: ID does not exist" containerID="a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c"
Jul 22 18:53:27.661068 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.661015    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c"} err="failed to get container status \"a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c\": rpc error: code = NotFound desc = could not find container \"a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c\": container with ID starting with a30d8904dca03529fbe2f8a4c640844ed9e5fc0f83f50b2df03bc367f7f8141c not found: ID does not exist"
Jul 22 18:53:27.673239 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.673216    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75d9999fb9-2f6z5"]
Jul 22 18:53:27.681400 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.681380    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75d9999fb9-2f6z5"]
Jul 22 18:53:27.687169 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:27.687151    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cdf4ef4-3348-47df-8922-d4704a175874" path="/var/lib/kubelet/pods/6cdf4ef4-3348-47df-8922-d4704a175874/volumes"
Jul 22 18:53:46.330192 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.330158    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"]
Jul 22 18:53:46.330618 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.330514    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdf4ef4-3348-47df-8922-d4704a175874" containerName="manager"
Jul 22 18:53:46.330618 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.330527    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdf4ef4-3348-47df-8922-d4704a175874" containerName="manager"
Jul 22 18:53:46.330618 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.330591    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdf4ef4-3348-47df-8922-d4704a175874" containerName="manager"
Jul 22 18:53:46.338401 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.338381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.341421 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.341399    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 22 18:53:46.341533 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.341405    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lkhjx\""
Jul 22 18:53:46.342863 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.342843    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"]
Jul 22 18:53:46.417315 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.417285    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06a46609-e9b0-4539-b0f2-446b3ddb7fc5-maas-api-tls\") pod \"maas-api-75bd6d7f99-msxr2\" (UID: \"06a46609-e9b0-4539-b0f2-446b3ddb7fc5\") " pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.417466 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.417372    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jskns\" (UniqueName: \"kubernetes.io/projected/06a46609-e9b0-4539-b0f2-446b3ddb7fc5-kube-api-access-jskns\") pod \"maas-api-75bd6d7f99-msxr2\" (UID: \"06a46609-e9b0-4539-b0f2-446b3ddb7fc5\") " pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.417466 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.417415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06a46609-e9b0-4539-b0f2-446b3ddb7fc5-openshift-service-ca\") pod \"maas-api-75bd6d7f99-msxr2\" (UID: \"06a46609-e9b0-4539-b0f2-446b3ddb7fc5\") " pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.518154 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.518128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06a46609-e9b0-4539-b0f2-446b3ddb7fc5-maas-api-tls\") pod \"maas-api-75bd6d7f99-msxr2\" (UID: \"06a46609-e9b0-4539-b0f2-446b3ddb7fc5\") " pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.518258 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.518187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jskns\" (UniqueName: \"kubernetes.io/projected/06a46609-e9b0-4539-b0f2-446b3ddb7fc5-kube-api-access-jskns\") pod \"maas-api-75bd6d7f99-msxr2\" (UID: \"06a46609-e9b0-4539-b0f2-446b3ddb7fc5\") " pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.518258 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.518213    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06a46609-e9b0-4539-b0f2-446b3ddb7fc5-openshift-service-ca\") pod \"maas-api-75bd6d7f99-msxr2\" (UID: \"06a46609-e9b0-4539-b0f2-446b3ddb7fc5\") " pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.518820 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.518802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06a46609-e9b0-4539-b0f2-446b3ddb7fc5-openshift-service-ca\") pod \"maas-api-75bd6d7f99-msxr2\" (UID: \"06a46609-e9b0-4539-b0f2-446b3ddb7fc5\") " pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.520682 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.520653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06a46609-e9b0-4539-b0f2-446b3ddb7fc5-maas-api-tls\") pod \"maas-api-75bd6d7f99-msxr2\" (UID: \"06a46609-e9b0-4539-b0f2-446b3ddb7fc5\") " pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.529132 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.529107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jskns\" (UniqueName: \"kubernetes.io/projected/06a46609-e9b0-4539-b0f2-446b3ddb7fc5-kube-api-access-jskns\") pod \"maas-api-75bd6d7f99-msxr2\" (UID: \"06a46609-e9b0-4539-b0f2-446b3ddb7fc5\") " pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.649938 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.649905    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:46.798162 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.798135    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"]
Jul 22 18:53:46.800908 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:53:46.800879    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a46609_e9b0_4539_b0f2_446b3ddb7fc5.slice/crio-f77aa6f57a7b83d94eb19a304fa2cafe7c5fcd4d0b286e79ef1ef1596f577a81 WatchSource:0}: Error finding container f77aa6f57a7b83d94eb19a304fa2cafe7c5fcd4d0b286e79ef1ef1596f577a81: Status 404 returned error can't find the container with id f77aa6f57a7b83d94eb19a304fa2cafe7c5fcd4d0b286e79ef1ef1596f577a81
Jul 22 18:53:46.924684 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.924592    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-p9785"]
Jul 22 18:53:46.929462 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.929441    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:53:46.934085 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.934060    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 22 18:53:46.934206 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.934150    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5kmmn\""
Jul 22 18:53:46.942802 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:46.942773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-p9785"]
Jul 22 18:53:47.022318 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.022287    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsh72\" (UniqueName: \"kubernetes.io/projected/099ade4d-ef1b-4d80-ab1c-8ba5a61f4946-kube-api-access-rsh72\") pod \"payload-processing-5bf47cd988-p9785\" (UID: \"099ade4d-ef1b-4d80-ab1c-8ba5a61f4946\") " pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:53:47.022485 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.022345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099ade4d-ef1b-4d80-ab1c-8ba5a61f4946-plugins-config-volume\") pod \"payload-processing-5bf47cd988-p9785\" (UID: \"099ade4d-ef1b-4d80-ab1c-8ba5a61f4946\") " pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:53:47.123580 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.123543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsh72\" (UniqueName: \"kubernetes.io/projected/099ade4d-ef1b-4d80-ab1c-8ba5a61f4946-kube-api-access-rsh72\") pod \"payload-processing-5bf47cd988-p9785\" (UID: \"099ade4d-ef1b-4d80-ab1c-8ba5a61f4946\") " pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:53:47.123758 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.123584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099ade4d-ef1b-4d80-ab1c-8ba5a61f4946-plugins-config-volume\") pod \"payload-processing-5bf47cd988-p9785\" (UID: \"099ade4d-ef1b-4d80-ab1c-8ba5a61f4946\") " pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:53:47.124251 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.124231    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099ade4d-ef1b-4d80-ab1c-8ba5a61f4946-plugins-config-volume\") pod \"payload-processing-5bf47cd988-p9785\" (UID: \"099ade4d-ef1b-4d80-ab1c-8ba5a61f4946\") " pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:53:47.131885 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.131843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsh72\" (UniqueName: \"kubernetes.io/projected/099ade4d-ef1b-4d80-ab1c-8ba5a61f4946-kube-api-access-rsh72\") pod \"payload-processing-5bf47cd988-p9785\" (UID: \"099ade4d-ef1b-4d80-ab1c-8ba5a61f4946\") " pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:53:47.240934 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.240853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:53:47.397906 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.397884    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-p9785"]
Jul 22 18:53:47.399916 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:53:47.399885    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099ade4d_ef1b_4d80_ab1c_8ba5a61f4946.slice/crio-acfbbef85c8918aac1e66be14da19dd6ebc003c07f9b1119e33064afd5ea4ae0 WatchSource:0}: Error finding container acfbbef85c8918aac1e66be14da19dd6ebc003c07f9b1119e33064afd5ea4ae0: Status 404 returned error can't find the container with id acfbbef85c8918aac1e66be14da19dd6ebc003c07f9b1119e33064afd5ea4ae0
Jul 22 18:53:47.760006 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.759964    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2" event={"ID":"06a46609-e9b0-4539-b0f2-446b3ddb7fc5","Type":"ContainerStarted","Data":"f77aa6f57a7b83d94eb19a304fa2cafe7c5fcd4d0b286e79ef1ef1596f577a81"}
Jul 22 18:53:47.762548 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:47.762515    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-p9785" event={"ID":"099ade4d-ef1b-4d80-ab1c-8ba5a61f4946","Type":"ContainerStarted","Data":"acfbbef85c8918aac1e66be14da19dd6ebc003c07f9b1119e33064afd5ea4ae0"}
Jul 22 18:53:49.776328 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:49.776288    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2" event={"ID":"06a46609-e9b0-4539-b0f2-446b3ddb7fc5","Type":"ContainerStarted","Data":"baa2fb5389fec6cb3a92920f001d3d3393ceecf652f5d8f22f312029c48943da"}
Jul 22 18:53:49.776763 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:49.776362    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:49.798172 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:49.798120    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2" podStartSLOduration=1.8752697870000001 podStartE2EDuration="3.798105232s" totalImagesPullingTime="1.922835445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:53:49.797754061 +0000 UTC m=+830.688295000" watchObservedRunningTime="2026-07-22 18:53:49.798105232 +0000 UTC m=+830.688646164"
Jul 22 18:53:50.784190 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:50.784150    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-p9785" event={"ID":"099ade4d-ef1b-4d80-ab1c-8ba5a61f4946","Type":"ContainerStarted","Data":"bfd32a5aca1bdb7e98b98b02b0f6e1bb2fa2681cf476be408470263f73e3d64c"}
Jul 22 18:53:50.784594 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:50.784236    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:53:50.809139 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:50.809089    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-p9785" podStartSLOduration=1.9191298959999998 podStartE2EDuration="4.809062305s" totalImagesPullingTime="2.889932409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:53:50.802011744 +0000 UTC m=+831.692552691" watchObservedRunningTime="2026-07-22 18:53:50.809062305 +0000 UTC m=+831.699603237"
Jul 22 18:53:54.804948 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:54.804870    2576 generic.go:356] "Generic (PLEG): container finished" podID="f6891e84-95bd-4474-bc17-43fd7b17f2c9" containerID="e9d07e899028c0c7ebd063ec52596809d71aaa454aee9d0c6c51c5e8c398609e" exitCode=0
Jul 22 18:53:54.805311 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:54.804942    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs" event={"ID":"f6891e84-95bd-4474-bc17-43fd7b17f2c9","Type":"ContainerDied","Data":"e9d07e899028c0c7ebd063ec52596809d71aaa454aee9d0c6c51c5e8c398609e"}
Jul 22 18:53:54.805351 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:54.805309    2576 scope.go:117] "RemoveContainer" containerID="e9d07e899028c0c7ebd063ec52596809d71aaa454aee9d0c6c51c5e8c398609e"
Jul 22 18:53:54.806266 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:54.806239    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:53:54.806321 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:54.806308    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:53:54.806369 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:54.806348    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 18:53:55.324340 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:55.324299    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:53:55.789333 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:55.789306    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-75bd6d7f99-msxr2"
Jul 22 18:53:55.812399 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:55.812368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs" event={"ID":"f6891e84-95bd-4474-bc17-43fd7b17f2c9","Type":"ContainerStarted","Data":"1f5fd5ab4747befb98ebac869dbe76be3a90a356fd76687694e173156d067bc3"}
Jul 22 18:53:56.183188 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:56.183148    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:53:56.184437 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:56.184408    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jrkgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 22 18:53:56.184559 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:56.184460    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs" podUID="f6891e84-95bd-4474-bc17-43fd7b17f2c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 22 18:53:57.183742 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:57.183698    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jrkgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 22 18:53:57.184157 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:57.183763    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs" podUID="f6891e84-95bd-4474-bc17-43fd7b17f2c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 22 18:53:58.183559 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:58.183518    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jrkgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 22 18:53:58.183733 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:58.183582    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs" podUID="f6891e84-95bd-4474-bc17-43fd7b17f2c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 22 18:53:59.183147 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:59.183108    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jrkgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 22 18:53:59.183615 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:53:59.183186    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs" podUID="f6891e84-95bd-4474-bc17-43fd7b17f2c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 22 18:54:00.187218 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:00.187184    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:54:00.187650 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:00.187449    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:54:00.188164 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:00.188147    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jrkgs"
Jul 22 18:54:01.790790 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:01.790762    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-p9785"
Jul 22 18:54:11.791763 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.791728    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"]
Jul 22 18:54:11.795413 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.795394    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.798134 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.798111    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 18:54:11.798231 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.798142    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 18:54:11.799425 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.799408    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-92cn2\""
Jul 22 18:54:11.799491 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.799478    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 22 18:54:11.802969 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.802948    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"]
Jul 22 18:54:11.865315 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.865282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.865484 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.865327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.865544 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.865481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.865544 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.865517    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2r7g\" (UniqueName: \"kubernetes.io/projected/6265cb97-fd13-46dd-b488-b25cec49b376-kube-api-access-d2r7g\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.865544 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.865539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.865734 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.865615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6265cb97-fd13-46dd-b488-b25cec49b376-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.967039 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.967000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.967223 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.967049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2r7g\" (UniqueName: \"kubernetes.io/projected/6265cb97-fd13-46dd-b488-b25cec49b376-kube-api-access-d2r7g\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.967223 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.967078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.967223 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.967121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6265cb97-fd13-46dd-b488-b25cec49b376-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.967223 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.967147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.967223 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.967173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.967615 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.967591    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.967815 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.967656    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.967815 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.967699    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.969645 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.969608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6265cb97-fd13-46dd-b488-b25cec49b376-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.969846 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.969827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6265cb97-fd13-46dd-b488-b25cec49b376-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:11.976547 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:11.976525    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2r7g\" (UniqueName: \"kubernetes.io/projected/6265cb97-fd13-46dd-b488-b25cec49b376-kube-api-access-d2r7g\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5nwlw\" (UID: \"6265cb97-fd13-46dd-b488-b25cec49b376\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:12.106505 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:12.106430    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:12.233867 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:12.233838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"]
Jul 22 18:54:12.236877 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:54:12.236848    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6265cb97_fd13_46dd_b488_b25cec49b376.slice/crio-e871ac82adfeb3e35fafa14835276d93d5d178bcf3ecd0770dff551f3d967f0b WatchSource:0}: Error finding container e871ac82adfeb3e35fafa14835276d93d5d178bcf3ecd0770dff551f3d967f0b: Status 404 returned error can't find the container with id e871ac82adfeb3e35fafa14835276d93d5d178bcf3ecd0770dff551f3d967f0b
Jul 22 18:54:12.894182 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:12.894096    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw" event={"ID":"6265cb97-fd13-46dd-b488-b25cec49b376","Type":"ContainerStarted","Data":"e871ac82adfeb3e35fafa14835276d93d5d178bcf3ecd0770dff551f3d967f0b"}
Jul 22 18:54:13.905797 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:13.905679    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw" event={"ID":"6265cb97-fd13-46dd-b488-b25cec49b376","Type":"ContainerStarted","Data":"122ba1fe4c1232c4acff914b47db7f5c6062dea05bc56a894739515ea147e4ec"}
Jul 22 18:54:13.925040 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:13.924991    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw" podStartSLOduration=1.88595276 podStartE2EDuration="2.924977088s" totalImagesPullingTime="1.039024328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:54:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:54:13.923598398 +0000 UTC m=+854.814139336" watchObservedRunningTime="2026-07-22 18:54:13.924977088 +0000 UTC m=+854.815518020"
Jul 22 18:54:22.106926 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:22.106895    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:22.107491 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:22.106982    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:22.119542 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:22.119520    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:22.958559 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:22.958530    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5nwlw"
Jul 22 18:54:44.193190 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.193156    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"]
Jul 22 18:54:44.198807 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.198786    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.201466 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.201444    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 22 18:54:44.205462 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.205440    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"]
Jul 22 18:54:44.337932 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.337901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.338100 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.337955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.338100 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.337979    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8de2e7b6-fc3f-4717-a685-50fc0e66512a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.338100 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.338042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsplx\" (UniqueName: \"kubernetes.io/projected/8de2e7b6-fc3f-4717-a685-50fc0e66512a-kube-api-access-tsplx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.338100 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.338097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.338239 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.338114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.439018 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.438981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.439186 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.439047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.439186 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.439075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8de2e7b6-fc3f-4717-a685-50fc0e66512a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.439186 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.439103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsplx\" (UniqueName: \"kubernetes.io/projected/8de2e7b6-fc3f-4717-a685-50fc0e66512a-kube-api-access-tsplx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.439186 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.439134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.439186 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.439159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.439445 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.439407    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.439564 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.439536    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.439674 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.439653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.441406 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.441382    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8de2e7b6-fc3f-4717-a685-50fc0e66512a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.441580 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.441562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8de2e7b6-fc3f-4717-a685-50fc0e66512a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.448448 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.448386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsplx\" (UniqueName: \"kubernetes.io/projected/8de2e7b6-fc3f-4717-a685-50fc0e66512a-kube-api-access-tsplx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds\" (UID: \"8de2e7b6-fc3f-4717-a685-50fc0e66512a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.511776 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.511748    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:44.648553 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:44.648524    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"]
Jul 22 18:54:44.650656 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:54:44.650613    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de2e7b6_fc3f_4717_a685_50fc0e66512a.slice/crio-7fc9f933b977dd3fd3cc65133ef62e71391fac5c6faedf60eae5d0fe01262ef8 WatchSource:0}: Error finding container 7fc9f933b977dd3fd3cc65133ef62e71391fac5c6faedf60eae5d0fe01262ef8: Status 404 returned error can't find the container with id 7fc9f933b977dd3fd3cc65133ef62e71391fac5c6faedf60eae5d0fe01262ef8
Jul 22 18:54:45.047916 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:45.047883    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds" event={"ID":"8de2e7b6-fc3f-4717-a685-50fc0e66512a","Type":"ContainerStarted","Data":"8a118918087602c78fe3e5d6af474ed5422a4003ffd5ca35ee152c6fbf82bdde"}
Jul 22 18:54:45.048075 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:45.047938    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds" event={"ID":"8de2e7b6-fc3f-4717-a685-50fc0e66512a","Type":"ContainerStarted","Data":"7fc9f933b977dd3fd3cc65133ef62e71391fac5c6faedf60eae5d0fe01262ef8"}
Jul 22 18:54:45.065946 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:45.065897    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds" podStartSLOduration=0.856099769 podStartE2EDuration="1.065881352s" totalImagesPullingTime="209.781583ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:54:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:54:45.065425207 +0000 UTC m=+885.955966138" watchObservedRunningTime="2026-07-22 18:54:45.065881352 +0000 UTC m=+885.956422283"
Jul 22 18:54:54.512225 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:54.512133    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:54.512225 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:54.512219    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:54.525149 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:54.525128    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:54:55.110192 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:54:55.110165    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds"
Jul 22 18:56:14.159689 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.159649    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-646cd68c8c-6j6lb"]
Jul 22 18:56:14.162945 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.162922    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-646cd68c8c-6j6lb"
Jul 22 18:56:14.170647 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.170604    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-646cd68c8c-6j6lb"]
Jul 22 18:56:14.283282 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.283248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnbg6\" (UniqueName: \"kubernetes.io/projected/e8a987da-fc08-4c4a-a741-98e4843d5986-kube-api-access-bnbg6\") pod \"authorino-646cd68c8c-6j6lb\" (UID: \"e8a987da-fc08-4c4a-a741-98e4843d5986\") " pod="kuadrant-system/authorino-646cd68c8c-6j6lb"
Jul 22 18:56:14.283415 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.283339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8a987da-fc08-4c4a-a741-98e4843d5986-tls-cert\") pod \"authorino-646cd68c8c-6j6lb\" (UID: \"e8a987da-fc08-4c4a-a741-98e4843d5986\") " pod="kuadrant-system/authorino-646cd68c8c-6j6lb"
Jul 22 18:56:14.384004 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.383976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnbg6\" (UniqueName: \"kubernetes.io/projected/e8a987da-fc08-4c4a-a741-98e4843d5986-kube-api-access-bnbg6\") pod \"authorino-646cd68c8c-6j6lb\" (UID: \"e8a987da-fc08-4c4a-a741-98e4843d5986\") " pod="kuadrant-system/authorino-646cd68c8c-6j6lb"
Jul 22 18:56:14.384127 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.384034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8a987da-fc08-4c4a-a741-98e4843d5986-tls-cert\") pod \"authorino-646cd68c8c-6j6lb\" (UID: \"e8a987da-fc08-4c4a-a741-98e4843d5986\") " pod="kuadrant-system/authorino-646cd68c8c-6j6lb"
Jul 22 18:56:14.386435 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.386417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8a987da-fc08-4c4a-a741-98e4843d5986-tls-cert\") pod \"authorino-646cd68c8c-6j6lb\" (UID: \"e8a987da-fc08-4c4a-a741-98e4843d5986\") " pod="kuadrant-system/authorino-646cd68c8c-6j6lb"
Jul 22 18:56:14.391509 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.391490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnbg6\" (UniqueName: \"kubernetes.io/projected/e8a987da-fc08-4c4a-a741-98e4843d5986-kube-api-access-bnbg6\") pod \"authorino-646cd68c8c-6j6lb\" (UID: \"e8a987da-fc08-4c4a-a741-98e4843d5986\") " pod="kuadrant-system/authorino-646cd68c8c-6j6lb"
Jul 22 18:56:14.472845 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.472780    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-646cd68c8c-6j6lb"
Jul 22 18:56:14.623745 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:56:14.623711    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8a987da_fc08_4c4a_a741_98e4843d5986.slice/crio-2de7ffdf1d5edbe0a669d355acd6a28cb5e7c7259733614891be8f3ebd8acb3e WatchSource:0}: Error finding container 2de7ffdf1d5edbe0a669d355acd6a28cb5e7c7259733614891be8f3ebd8acb3e: Status 404 returned error can't find the container with id 2de7ffdf1d5edbe0a669d355acd6a28cb5e7c7259733614891be8f3ebd8acb3e
Jul 22 18:56:14.625075 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.625047    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-646cd68c8c-6j6lb"]
Jul 22 18:56:14.625075 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:14.625070    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 18:56:15.478460 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.478373    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-646cd68c8c-6j6lb" event={"ID":"e8a987da-fc08-4c4a-a741-98e4843d5986","Type":"ContainerStarted","Data":"e046814124adbdf58dd539ee8abd6cdfa7a71ca0afb84be0f321bf182491b1fe"}
Jul 22 18:56:15.478460 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.478415    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-646cd68c8c-6j6lb" event={"ID":"e8a987da-fc08-4c4a-a741-98e4843d5986","Type":"ContainerStarted","Data":"2de7ffdf1d5edbe0a669d355acd6a28cb5e7c7259733614891be8f3ebd8acb3e"}
Jul 22 18:56:15.501004 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.500938    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-646cd68c8c-6j6lb" podStartSLOduration=0.985620013 podStartE2EDuration="1.500915473s" totalImagesPullingTime="515.29546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:15.498531326 +0000 UTC m=+976.389072259" watchObservedRunningTime="2026-07-22 18:56:15.500915473 +0000 UTC m=+976.391456409"
Jul 22 18:56:15.534245 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.534206    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67cf47bbfd-sm9lg"]
Jul 22 18:56:15.534465 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.534435    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg" podUID="eb996e96-8754-4ef6-90b3-e41c4604319a" containerName="authorino" containerID="cri-o://ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211" gracePeriod=30
Jul 22 18:56:15.804500 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.804471    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:56:15.897759 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.897729    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/eb996e96-8754-4ef6-90b3-e41c4604319a-tls-cert\") pod \"eb996e96-8754-4ef6-90b3-e41c4604319a\" (UID: \"eb996e96-8754-4ef6-90b3-e41c4604319a\") "
Jul 22 18:56:15.897957 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.897804    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djvrw\" (UniqueName: \"kubernetes.io/projected/eb996e96-8754-4ef6-90b3-e41c4604319a-kube-api-access-djvrw\") pod \"eb996e96-8754-4ef6-90b3-e41c4604319a\" (UID: \"eb996e96-8754-4ef6-90b3-e41c4604319a\") "
Jul 22 18:56:15.899833 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.899798    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb996e96-8754-4ef6-90b3-e41c4604319a-kube-api-access-djvrw" (OuterVolumeSpecName: "kube-api-access-djvrw") pod "eb996e96-8754-4ef6-90b3-e41c4604319a" (UID: "eb996e96-8754-4ef6-90b3-e41c4604319a"). InnerVolumeSpecName "kube-api-access-djvrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:56:15.908566 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.908543    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb996e96-8754-4ef6-90b3-e41c4604319a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "eb996e96-8754-4ef6-90b3-e41c4604319a" (UID: "eb996e96-8754-4ef6-90b3-e41c4604319a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:56:15.998949 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.998917    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/eb996e96-8754-4ef6-90b3-e41c4604319a-tls-cert\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:56:15.998949 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:15.998946    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djvrw\" (UniqueName: \"kubernetes.io/projected/eb996e96-8754-4ef6-90b3-e41c4604319a-kube-api-access-djvrw\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:56:16.485543 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:16.485505    2576 generic.go:356] "Generic (PLEG): container finished" podID="eb996e96-8754-4ef6-90b3-e41c4604319a" containerID="ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211" exitCode=0
Jul 22 18:56:16.486021 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:16.485552    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg"
Jul 22 18:56:16.486021 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:16.485593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg" event={"ID":"eb996e96-8754-4ef6-90b3-e41c4604319a","Type":"ContainerDied","Data":"ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211"}
Jul 22 18:56:16.486021 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:16.485658    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67cf47bbfd-sm9lg" event={"ID":"eb996e96-8754-4ef6-90b3-e41c4604319a","Type":"ContainerDied","Data":"168fd0decdbe0759f149147af4fcb7b81df233312590b1c979039e488304d11e"}
Jul 22 18:56:16.486021 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:16.485677    2576 scope.go:117] "RemoveContainer" containerID="ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211"
Jul 22 18:56:16.496274 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:16.496254    2576 scope.go:117] "RemoveContainer" containerID="ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211"
Jul 22 18:56:16.496552 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:56:16.496530    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211\": container with ID starting with ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211 not found: ID does not exist" containerID="ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211"
Jul 22 18:56:16.496656 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:16.496565    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211"} err="failed to get container status \"ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211\": rpc error: code = NotFound desc = could not find container \"ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211\": container with ID starting with ed2687eec65e3c432056df863f279a8eb9d72b4247c46b69ab1145759621d211 not found: ID does not exist"
Jul 22 18:56:16.516251 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:16.516222    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67cf47bbfd-sm9lg"]
Jul 22 18:56:16.520045 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:16.520018    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-67cf47bbfd-sm9lg"]
Jul 22 18:56:17.689289 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:56:17.689252    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb996e96-8754-4ef6-90b3-e41c4604319a" path="/var/lib/kubelet/pods/eb996e96-8754-4ef6-90b3-e41c4604319a/volumes"
Jul 22 18:57:02.265307 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.265265    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9d64f8564-wqbqd"]
Jul 22 18:57:02.265749 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.265474    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-9d64f8564-wqbqd" podUID="36f7ae02-7fbe-437c-be3d-b140c8477a48" containerName="manager" containerID="cri-o://7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f" gracePeriod=30
Jul 22 18:57:02.513366 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.513344    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:57:02.590715 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.590617    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36f7ae02-7fbe-437c-be3d-b140c8477a48-webhook-cert\") pod \"36f7ae02-7fbe-437c-be3d-b140c8477a48\" (UID: \"36f7ae02-7fbe-437c-be3d-b140c8477a48\") "
Jul 22 18:57:02.590715 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.590680    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmttj\" (UniqueName: \"kubernetes.io/projected/36f7ae02-7fbe-437c-be3d-b140c8477a48-kube-api-access-dmttj\") pod \"36f7ae02-7fbe-437c-be3d-b140c8477a48\" (UID: \"36f7ae02-7fbe-437c-be3d-b140c8477a48\") "
Jul 22 18:57:02.592620 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.592588    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f7ae02-7fbe-437c-be3d-b140c8477a48-kube-api-access-dmttj" (OuterVolumeSpecName: "kube-api-access-dmttj") pod "36f7ae02-7fbe-437c-be3d-b140c8477a48" (UID: "36f7ae02-7fbe-437c-be3d-b140c8477a48"). InnerVolumeSpecName "kube-api-access-dmttj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:57:02.592746 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.592725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36f7ae02-7fbe-437c-be3d-b140c8477a48-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "36f7ae02-7fbe-437c-be3d-b140c8477a48" (UID: "36f7ae02-7fbe-437c-be3d-b140c8477a48"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:57:02.691379 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.691351    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36f7ae02-7fbe-437c-be3d-b140c8477a48-webhook-cert\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:57:02.691379 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.691375    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmttj\" (UniqueName: \"kubernetes.io/projected/36f7ae02-7fbe-437c-be3d-b140c8477a48-kube-api-access-dmttj\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:57:02.707497 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.707470    2576 generic.go:356] "Generic (PLEG): container finished" podID="36f7ae02-7fbe-437c-be3d-b140c8477a48" containerID="7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f" exitCode=0
Jul 22 18:57:02.707652 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.707511    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-wqbqd"
Jul 22 18:57:02.707652 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.707547    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-wqbqd" event={"ID":"36f7ae02-7fbe-437c-be3d-b140c8477a48","Type":"ContainerDied","Data":"7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f"}
Jul 22 18:57:02.707652 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.707600    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-wqbqd" event={"ID":"36f7ae02-7fbe-437c-be3d-b140c8477a48","Type":"ContainerDied","Data":"43666ae40e26d5d0c2cdd21a1b7f11a58744ee3fd41ab3b30dc3cfa958d451a3"}
Jul 22 18:57:02.707838 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.707653    2576 scope.go:117] "RemoveContainer" containerID="7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f"
Jul 22 18:57:02.719431 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.719415    2576 scope.go:117] "RemoveContainer" containerID="7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f"
Jul 22 18:57:02.719708 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:57:02.719687    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f\": container with ID starting with 7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f not found: ID does not exist" containerID="7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f"
Jul 22 18:57:02.719784 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.719717    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f"} err="failed to get container status \"7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f\": rpc error: code = NotFound desc = could not find container \"7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f\": container with ID starting with 7aae7edb8ef29935ab3dd85c02cb09224d4f52c39f690567cd3f634a653c691f not found: ID does not exist"
Jul 22 18:57:02.731407 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.731381    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9d64f8564-wqbqd"]
Jul 22 18:57:02.733903 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:02.733883    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9d64f8564-wqbqd"]
Jul 22 18:57:03.693037 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:57:03.693005    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36f7ae02-7fbe-437c-be3d-b140c8477a48" path="/var/lib/kubelet/pods/36f7ae02-7fbe-437c-be3d-b140c8477a48/volumes"
Jul 22 18:58:03.597082 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.597040    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9d64f8564-tzpps"]
Jul 22 18:58:03.597464 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.597380    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb996e96-8754-4ef6-90b3-e41c4604319a" containerName="authorino"
Jul 22 18:58:03.597464 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.597391    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb996e96-8754-4ef6-90b3-e41c4604319a" containerName="authorino"
Jul 22 18:58:03.597464 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.597415    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f7ae02-7fbe-437c-be3d-b140c8477a48" containerName="manager"
Jul 22 18:58:03.597464 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.597421    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f7ae02-7fbe-437c-be3d-b140c8477a48" containerName="manager"
Jul 22 18:58:03.597589 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.597476    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb996e96-8754-4ef6-90b3-e41c4604319a" containerName="authorino"
Jul 22 18:58:03.597589 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.597484    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="36f7ae02-7fbe-437c-be3d-b140c8477a48" containerName="manager"
Jul 22 18:58:03.600450 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.600433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:03.604442 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.604412    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 18:58:03.604442 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.604426    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 18:58:03.604618 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.604478    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rv8cf\""
Jul 22 18:58:03.610410 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.610383    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9d64f8564-tzpps"]
Jul 22 18:58:03.706888 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.706856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmc7m\" (UniqueName: \"kubernetes.io/projected/c30e7397-865d-4a6e-b649-1029ece964e9-kube-api-access-kmc7m\") pod \"maas-controller-9d64f8564-tzpps\" (UID: \"c30e7397-865d-4a6e-b649-1029ece964e9\") " pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:03.707034 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.706904    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30e7397-865d-4a6e-b649-1029ece964e9-webhook-cert\") pod \"maas-controller-9d64f8564-tzpps\" (UID: \"c30e7397-865d-4a6e-b649-1029ece964e9\") " pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:03.807847 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.807812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmc7m\" (UniqueName: \"kubernetes.io/projected/c30e7397-865d-4a6e-b649-1029ece964e9-kube-api-access-kmc7m\") pod \"maas-controller-9d64f8564-tzpps\" (UID: \"c30e7397-865d-4a6e-b649-1029ece964e9\") " pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:03.807847 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.807855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30e7397-865d-4a6e-b649-1029ece964e9-webhook-cert\") pod \"maas-controller-9d64f8564-tzpps\" (UID: \"c30e7397-865d-4a6e-b649-1029ece964e9\") " pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:03.810299 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.810275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30e7397-865d-4a6e-b649-1029ece964e9-webhook-cert\") pod \"maas-controller-9d64f8564-tzpps\" (UID: \"c30e7397-865d-4a6e-b649-1029ece964e9\") " pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:03.816091 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.816069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmc7m\" (UniqueName: \"kubernetes.io/projected/c30e7397-865d-4a6e-b649-1029ece964e9-kube-api-access-kmc7m\") pod \"maas-controller-9d64f8564-tzpps\" (UID: \"c30e7397-865d-4a6e-b649-1029ece964e9\") " pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:03.911553 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:03.911524    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:04.042247 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:04.042123    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9d64f8564-tzpps"]
Jul 22 18:58:04.044953 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:58:04.044924    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30e7397_865d_4a6e_b649_1029ece964e9.slice/crio-584c3e6cbf27e95eef73ae6e33b0801b8b87e726fd8f8d1fbd218673b9551272 WatchSource:0}: Error finding container 584c3e6cbf27e95eef73ae6e33b0801b8b87e726fd8f8d1fbd218673b9551272: Status 404 returned error can't find the container with id 584c3e6cbf27e95eef73ae6e33b0801b8b87e726fd8f8d1fbd218673b9551272
Jul 22 18:58:04.993063 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:04.993026    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-tzpps" event={"ID":"c30e7397-865d-4a6e-b649-1029ece964e9","Type":"ContainerStarted","Data":"14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117"}
Jul 22 18:58:04.993452 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:04.993087    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-tzpps" event={"ID":"c30e7397-865d-4a6e-b649-1029ece964e9","Type":"ContainerStarted","Data":"584c3e6cbf27e95eef73ae6e33b0801b8b87e726fd8f8d1fbd218673b9551272"}
Jul 22 18:58:04.993452 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:04.993138    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:05.010940 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:05.010896    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9d64f8564-tzpps" podStartSLOduration=1.640808327 podStartE2EDuration="2.010883673s" totalImagesPullingTime="370.075346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:58:05.009895628 +0000 UTC m=+1085.900436559" watchObservedRunningTime="2026-07-22 18:58:05.010883673 +0000 UTC m=+1085.901424605"
Jul 22 18:58:15.999757 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:15.999724    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:20.702996 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:20.702954    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9d64f8564-tzpps"]
Jul 22 18:58:20.703461 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:20.703201    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-9d64f8564-tzpps" podUID="c30e7397-865d-4a6e-b649-1029ece964e9" containerName="manager" containerID="cri-o://14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117" gracePeriod=30
Jul 22 18:58:20.957003 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:20.956950    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:21.060541 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.060509    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmc7m\" (UniqueName: \"kubernetes.io/projected/c30e7397-865d-4a6e-b649-1029ece964e9-kube-api-access-kmc7m\") pod \"c30e7397-865d-4a6e-b649-1029ece964e9\" (UID: \"c30e7397-865d-4a6e-b649-1029ece964e9\") "
Jul 22 18:58:21.060541 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.060544    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30e7397-865d-4a6e-b649-1029ece964e9-webhook-cert\") pod \"c30e7397-865d-4a6e-b649-1029ece964e9\" (UID: \"c30e7397-865d-4a6e-b649-1029ece964e9\") "
Jul 22 18:58:21.062522 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.062499    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c30e7397-865d-4a6e-b649-1029ece964e9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c30e7397-865d-4a6e-b649-1029ece964e9" (UID: "c30e7397-865d-4a6e-b649-1029ece964e9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:58:21.062651 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.062554    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30e7397-865d-4a6e-b649-1029ece964e9-kube-api-access-kmc7m" (OuterVolumeSpecName: "kube-api-access-kmc7m") pod "c30e7397-865d-4a6e-b649-1029ece964e9" (UID: "c30e7397-865d-4a6e-b649-1029ece964e9"). InnerVolumeSpecName "kube-api-access-kmc7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:58:21.069974 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.069945    2576 generic.go:356] "Generic (PLEG): container finished" podID="c30e7397-865d-4a6e-b649-1029ece964e9" containerID="14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117" exitCode=0
Jul 22 18:58:21.070078 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.069988    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-tzpps"
Jul 22 18:58:21.070078 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.070032    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-tzpps" event={"ID":"c30e7397-865d-4a6e-b649-1029ece964e9","Type":"ContainerDied","Data":"14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117"}
Jul 22 18:58:21.070187 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.070091    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-tzpps" event={"ID":"c30e7397-865d-4a6e-b649-1029ece964e9","Type":"ContainerDied","Data":"584c3e6cbf27e95eef73ae6e33b0801b8b87e726fd8f8d1fbd218673b9551272"}
Jul 22 18:58:21.070187 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.070123    2576 scope.go:117] "RemoveContainer" containerID="14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117"
Jul 22 18:58:21.078305 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.078288    2576 scope.go:117] "RemoveContainer" containerID="14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117"
Jul 22 18:58:21.078596 ip-10-0-139-185 kubenswrapper[2576]: E0722 18:58:21.078576    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117\": container with ID starting with 14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117 not found: ID does not exist" containerID="14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117"
Jul 22 18:58:21.078697 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.078603    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117"} err="failed to get container status \"14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117\": rpc error: code = NotFound desc = could not find container \"14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117\": container with ID starting with 14d0c2d7d3232c56ce276ae49356ce38e4d84da76aabacffe16c2fef7a9d8117 not found: ID does not exist"
Jul 22 18:58:21.092025 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.091980    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9d64f8564-tzpps"]
Jul 22 18:58:21.093760 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.093740    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9d64f8564-tzpps"]
Jul 22 18:58:21.161579 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.161546    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmc7m\" (UniqueName: \"kubernetes.io/projected/c30e7397-865d-4a6e-b649-1029ece964e9-kube-api-access-kmc7m\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:58:21.161579 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.161575    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30e7397-865d-4a6e-b649-1029ece964e9-webhook-cert\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 18:58:21.688597 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.688558    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30e7397-865d-4a6e-b649-1029ece964e9" path="/var/lib/kubelet/pods/c30e7397-865d-4a6e-b649-1029ece964e9/volumes"
Jul 22 18:58:21.910498 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.910463    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9d64f8564-pggwv"]
Jul 22 18:58:21.910881 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.910851    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30e7397-865d-4a6e-b649-1029ece964e9" containerName="manager"
Jul 22 18:58:21.910881 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.910866    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30e7397-865d-4a6e-b649-1029ece964e9" containerName="manager"
Jul 22 18:58:21.914735 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.911365    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30e7397-865d-4a6e-b649-1029ece964e9" containerName="manager"
Jul 22 18:58:21.917282 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.917249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 18:58:21.920351 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.920327    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 18:58:21.920351 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.920338    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 18:58:21.921522 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.921503    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rv8cf\""
Jul 22 18:58:21.926045 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:21.926022    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9d64f8564-pggwv"]
Jul 22 18:58:22.069177 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:22.069102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/630835cd-bd64-4087-9cca-b3d042fb2c1e-webhook-cert\") pod \"maas-controller-9d64f8564-pggwv\" (UID: \"630835cd-bd64-4087-9cca-b3d042fb2c1e\") " pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 18:58:22.069177 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:22.069139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45qvq\" (UniqueName: \"kubernetes.io/projected/630835cd-bd64-4087-9cca-b3d042fb2c1e-kube-api-access-45qvq\") pod \"maas-controller-9d64f8564-pggwv\" (UID: \"630835cd-bd64-4087-9cca-b3d042fb2c1e\") " pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 18:58:22.169981 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:22.169934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/630835cd-bd64-4087-9cca-b3d042fb2c1e-webhook-cert\") pod \"maas-controller-9d64f8564-pggwv\" (UID: \"630835cd-bd64-4087-9cca-b3d042fb2c1e\") " pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 18:58:22.169981 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:22.169977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45qvq\" (UniqueName: \"kubernetes.io/projected/630835cd-bd64-4087-9cca-b3d042fb2c1e-kube-api-access-45qvq\") pod \"maas-controller-9d64f8564-pggwv\" (UID: \"630835cd-bd64-4087-9cca-b3d042fb2c1e\") " pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 18:58:22.172288 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:22.172258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/630835cd-bd64-4087-9cca-b3d042fb2c1e-webhook-cert\") pod \"maas-controller-9d64f8564-pggwv\" (UID: \"630835cd-bd64-4087-9cca-b3d042fb2c1e\") " pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 18:58:22.180569 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:22.180538    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45qvq\" (UniqueName: \"kubernetes.io/projected/630835cd-bd64-4087-9cca-b3d042fb2c1e-kube-api-access-45qvq\") pod \"maas-controller-9d64f8564-pggwv\" (UID: \"630835cd-bd64-4087-9cca-b3d042fb2c1e\") " pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 18:58:22.228755 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:22.228726    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 18:58:22.361160 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:22.361131    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9d64f8564-pggwv"]
Jul 22 18:58:22.362890 ip-10-0-139-185 kubenswrapper[2576]: W0722 18:58:22.362862    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod630835cd_bd64_4087_9cca_b3d042fb2c1e.slice/crio-79cecc8114c69ee1f9e1752380603d6ac4ab4b6b314a991ee2240c17d85090a0 WatchSource:0}: Error finding container 79cecc8114c69ee1f9e1752380603d6ac4ab4b6b314a991ee2240c17d85090a0: Status 404 returned error can't find the container with id 79cecc8114c69ee1f9e1752380603d6ac4ab4b6b314a991ee2240c17d85090a0
Jul 22 18:58:23.081652 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:23.081547    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-pggwv" event={"ID":"630835cd-bd64-4087-9cca-b3d042fb2c1e","Type":"ContainerStarted","Data":"4ef52683037c52ec1d515681f2745e723bd4218d41b10421c44faa2501fb09cd"}
Jul 22 18:58:23.081652 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:23.081601    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9d64f8564-pggwv" event={"ID":"630835cd-bd64-4087-9cca-b3d042fb2c1e","Type":"ContainerStarted","Data":"79cecc8114c69ee1f9e1752380603d6ac4ab4b6b314a991ee2240c17d85090a0"}
Jul 22 18:58:23.082044 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:23.081660    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 18:58:23.101313 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:23.101267    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9d64f8564-pggwv" podStartSLOduration=1.749525567 podStartE2EDuration="2.101253115s" totalImagesPullingTime="351.727548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:58:23.099922845 +0000 UTC m=+1103.990463787" watchObservedRunningTime="2026-07-22 18:58:23.101253115 +0000 UTC m=+1103.991794046"
Jul 22 18:58:34.088751 ip-10-0-139-185 kubenswrapper[2576]: I0722 18:58:34.088719    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9d64f8564-pggwv"
Jul 22 19:03:01.619047 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:01.619009    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"]
Jul 22 19:03:01.619682 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:01.619261    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc" podUID="ca648b81-0fef-4dc5-a239-88b70c12dd8b" containerName="manager" containerID="cri-o://987453b707058432eed159c4c0c1573afec30852f9af3e81a0e2683fb644ca97" gracePeriod=10
Jul 22 19:03:05.418470 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:05.418440    2576 generic.go:356] "Generic (PLEG): container finished" podID="ca648b81-0fef-4dc5-a239-88b70c12dd8b" containerID="987453b707058432eed159c4c0c1573afec30852f9af3e81a0e2683fb644ca97" exitCode=0
Jul 22 19:03:05.418813 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:05.418513    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc" event={"ID":"ca648b81-0fef-4dc5-a239-88b70c12dd8b","Type":"ContainerDied","Data":"987453b707058432eed159c4c0c1573afec30852f9af3e81a0e2683fb644ca97"}
Jul 22 19:03:05.469661 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:05.469639    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 19:03:05.628273 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:05.628246    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmbwd\" (UniqueName: \"kubernetes.io/projected/ca648b81-0fef-4dc5-a239-88b70c12dd8b-kube-api-access-xmbwd\") pod \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\" (UID: \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\") "
Jul 22 19:03:05.628439 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:05.628285    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca648b81-0fef-4dc5-a239-88b70c12dd8b-extensions-socket-volume\") pod \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\" (UID: \"ca648b81-0fef-4dc5-a239-88b70c12dd8b\") "
Jul 22 19:03:05.628722 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:05.628698    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca648b81-0fef-4dc5-a239-88b70c12dd8b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ca648b81-0fef-4dc5-a239-88b70c12dd8b" (UID: "ca648b81-0fef-4dc5-a239-88b70c12dd8b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:03:05.630332 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:05.630312    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca648b81-0fef-4dc5-a239-88b70c12dd8b-kube-api-access-xmbwd" (OuterVolumeSpecName: "kube-api-access-xmbwd") pod "ca648b81-0fef-4dc5-a239-88b70c12dd8b" (UID: "ca648b81-0fef-4dc5-a239-88b70c12dd8b"). InnerVolumeSpecName "kube-api-access-xmbwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:03:05.729424 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:05.729399    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmbwd\" (UniqueName: \"kubernetes.io/projected/ca648b81-0fef-4dc5-a239-88b70c12dd8b-kube-api-access-xmbwd\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:03:05.729524 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:05.729427    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca648b81-0fef-4dc5-a239-88b70c12dd8b-extensions-socket-volume\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:03:06.424057 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:06.424026    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"
Jul 22 19:03:06.424522 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:06.424061    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc" event={"ID":"ca648b81-0fef-4dc5-a239-88b70c12dd8b","Type":"ContainerDied","Data":"c271d518b63d848661dfaa04fa207033337d6b91b8d3f0a5a3fa619316b62738"}
Jul 22 19:03:06.424522 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:06.424122    2576 scope.go:117] "RemoveContainer" containerID="987453b707058432eed159c4c0c1573afec30852f9af3e81a0e2683fb644ca97"
Jul 22 19:03:06.444034 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:06.444011    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"]
Jul 22 19:03:06.449689 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:06.449664    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w2rqc"]
Jul 22 19:03:07.694238 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:03:07.694195    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca648b81-0fef-4dc5-a239-88b70c12dd8b" path="/var/lib/kubelet/pods/ca648b81-0fef-4dc5-a239-88b70c12dd8b/volumes"
Jul 22 19:04:02.838084 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:02.838044    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"]
Jul 22 19:04:02.838458 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:02.838440    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca648b81-0fef-4dc5-a239-88b70c12dd8b" containerName="manager"
Jul 22 19:04:02.838458 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:02.838454    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca648b81-0fef-4dc5-a239-88b70c12dd8b" containerName="manager"
Jul 22 19:04:02.838531 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:02.838521    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca648b81-0fef-4dc5-a239-88b70c12dd8b" containerName="manager"
Jul 22 19:04:02.841939 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:02.841923    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:04:02.846820 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:02.846796    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rn8nc\""
Jul 22 19:04:02.861941 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:02.861919    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"]
Jul 22 19:04:03.008977 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.008945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9febba1d-e8fe-478c-95fb-28542fac8300-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zst2d\" (UID: \"9febba1d-e8fe-478c-95fb-28542fac8300\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:04:03.009136 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.009006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkzlh\" (UniqueName: \"kubernetes.io/projected/9febba1d-e8fe-478c-95fb-28542fac8300-kube-api-access-rkzlh\") pod \"kuadrant-operator-controller-manager-86878c769f-zst2d\" (UID: \"9febba1d-e8fe-478c-95fb-28542fac8300\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:04:03.109850 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.109774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9febba1d-e8fe-478c-95fb-28542fac8300-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zst2d\" (UID: \"9febba1d-e8fe-478c-95fb-28542fac8300\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:04:03.109850 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.109839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkzlh\" (UniqueName: \"kubernetes.io/projected/9febba1d-e8fe-478c-95fb-28542fac8300-kube-api-access-rkzlh\") pod \"kuadrant-operator-controller-manager-86878c769f-zst2d\" (UID: \"9febba1d-e8fe-478c-95fb-28542fac8300\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:04:03.110131 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.110113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9febba1d-e8fe-478c-95fb-28542fac8300-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zst2d\" (UID: \"9febba1d-e8fe-478c-95fb-28542fac8300\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:04:03.118533 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.118504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkzlh\" (UniqueName: \"kubernetes.io/projected/9febba1d-e8fe-478c-95fb-28542fac8300-kube-api-access-rkzlh\") pod \"kuadrant-operator-controller-manager-86878c769f-zst2d\" (UID: \"9febba1d-e8fe-478c-95fb-28542fac8300\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:04:03.151549 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.151528    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:04:03.287040 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.287013    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"]
Jul 22 19:04:03.289439 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:04:03.289414    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9febba1d_e8fe_478c_95fb_28542fac8300.slice/crio-43e388d65878d1b589ac0d2149c4623bebc0ef404cf5889e0d2efcd0a582ea73 WatchSource:0}: Error finding container 43e388d65878d1b589ac0d2149c4623bebc0ef404cf5889e0d2efcd0a582ea73: Status 404 returned error can't find the container with id 43e388d65878d1b589ac0d2149c4623bebc0ef404cf5889e0d2efcd0a582ea73
Jul 22 19:04:03.690341 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.690303    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d" event={"ID":"9febba1d-e8fe-478c-95fb-28542fac8300","Type":"ContainerStarted","Data":"3526ac0846e59a9fadf71c7a6d10f4cdcb4cbf6b23d820e707396180d402c9b3"}
Jul 22 19:04:03.690520 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.690358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d" event={"ID":"9febba1d-e8fe-478c-95fb-28542fac8300","Type":"ContainerStarted","Data":"43e388d65878d1b589ac0d2149c4623bebc0ef404cf5889e0d2efcd0a582ea73"}
Jul 22 19:04:03.690520 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.690395    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:04:03.711592 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:03.711544    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d" podStartSLOduration=1.7115294890000001 podStartE2EDuration="1.711529489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:04:03.710034752 +0000 UTC m=+1444.600575683" watchObservedRunningTime="2026-07-22 19:04:03.711529489 +0000 UTC m=+1444.602070421"
Jul 22 19:04:14.698051 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:04:14.698016    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zst2d"
Jul 22 19:06:55.772560 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.772471    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"]
Jul 22 19:06:55.775872 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.775851    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:55.778812 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.778789    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-df776eca-kserve-self-signed-certs\""
Jul 22 19:06:55.794835 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.794800    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"]
Jul 22 19:06:55.965706 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.965672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-home\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:55.965865 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.965771    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsss9\" (UniqueName: \"kubernetes.io/projected/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-kube-api-access-wsss9\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:55.965865 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.965808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tls-certs\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:55.965865 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.965848    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-dshm\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:55.965972 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.965867    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tmp-dir\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:55.965972 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:55.965922    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-model-cache\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.067372 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.067286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-dshm\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.067372 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.067344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tmp-dir\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.067664 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.067392    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-model-cache\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.067664 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.067440    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-home\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.067664 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.067483    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsss9\" (UniqueName: \"kubernetes.io/projected/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-kube-api-access-wsss9\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.067664 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.067508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tls-certs\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.067891 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.067838    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tmp-dir\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.067891 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.067847    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-model-cache\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.068114 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.068092    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-home\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.069814 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.069784    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-dshm\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.070204 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.070187    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tls-certs\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.077780 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.077752    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsss9\" (UniqueName: \"kubernetes.io/projected/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-kube-api-access-wsss9\") pod \"e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") " pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.088376 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.088348    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:06:56.220592 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.220519    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"]
Jul 22 19:06:56.224039 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:06:56.224005    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96c031cb_ffcf_435e_bfe6_3cc4ddf27295.slice/crio-93bf7f48e6f7b390f294c3fd05f765ddc73186a6a5ee41414bb4400906546770 WatchSource:0}: Error finding container 93bf7f48e6f7b390f294c3fd05f765ddc73186a6a5ee41414bb4400906546770: Status 404 returned error can't find the container with id 93bf7f48e6f7b390f294c3fd05f765ddc73186a6a5ee41414bb4400906546770
Jul 22 19:06:56.508614 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.508569    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z" event={"ID":"96c031cb-ffcf-435e-bfe6-3cc4ddf27295","Type":"ContainerStarted","Data":"eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968"}
Jul 22 19:06:56.508788 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.508636    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z" event={"ID":"96c031cb-ffcf-435e-bfe6-3cc4ddf27295","Type":"ContainerStarted","Data":"93bf7f48e6f7b390f294c3fd05f765ddc73186a6a5ee41414bb4400906546770"}
Jul 22 19:06:56.527257 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:06:56.527207    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z" podStartSLOduration=1.5271934790000001 podStartE2EDuration="1.527193479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:06:56.525245713 +0000 UTC m=+1617.415786646" watchObservedRunningTime="2026-07-22 19:06:56.527193479 +0000 UTC m=+1617.417734411"
Jul 22 19:07:06.088928 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:07:06.088895    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:07:06.089346 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:07:06.088965    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:07:06.101500 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:07:06.101471    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:07:06.570249 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:07:06.570224    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:07:45.907602 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:07:45.907568    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"]
Jul 22 19:08:00.928233 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:00.928173    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z" podUID="96c031cb-ffcf-435e-bfe6-3cc4ddf27295" containerName="main" containerID="cri-o://eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968" gracePeriod=45
Jul 22 19:08:01.177382 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.177352    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:08:01.220221 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220143    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsss9\" (UniqueName: \"kubernetes.io/projected/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-kube-api-access-wsss9\") pod \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") "
Jul 22 19:08:01.220221 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220198    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tls-certs\") pod \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") "
Jul 22 19:08:01.220474 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220236    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-home\") pod \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") "
Jul 22 19:08:01.220474 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220375    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-dshm\") pod \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") "
Jul 22 19:08:01.220474 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220416    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-model-cache\") pod \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") "
Jul 22 19:08:01.220474 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220473    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tmp-dir\") pod \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\" (UID: \"96c031cb-ffcf-435e-bfe6-3cc4ddf27295\") "
Jul 22 19:08:01.220758 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220495    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-home" (OuterVolumeSpecName: "home") pod "96c031cb-ffcf-435e-bfe6-3cc4ddf27295" (UID: "96c031cb-ffcf-435e-bfe6-3cc4ddf27295"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:08:01.220758 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220721    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-model-cache" (OuterVolumeSpecName: "model-cache") pod "96c031cb-ffcf-435e-bfe6-3cc4ddf27295" (UID: "96c031cb-ffcf-435e-bfe6-3cc4ddf27295"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:08:01.221523 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220888    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-home\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:08:01.221523 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220904    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-model-cache\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:08:01.221523 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.220921    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "96c031cb-ffcf-435e-bfe6-3cc4ddf27295" (UID: "96c031cb-ffcf-435e-bfe6-3cc4ddf27295"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:08:01.222915 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.222872    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-dshm" (OuterVolumeSpecName: "dshm") pod "96c031cb-ffcf-435e-bfe6-3cc4ddf27295" (UID: "96c031cb-ffcf-435e-bfe6-3cc4ddf27295"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:08:01.222915 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.222893    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-kube-api-access-wsss9" (OuterVolumeSpecName: "kube-api-access-wsss9") pod "96c031cb-ffcf-435e-bfe6-3cc4ddf27295" (UID: "96c031cb-ffcf-435e-bfe6-3cc4ddf27295"). InnerVolumeSpecName "kube-api-access-wsss9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:08:01.223046 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.222949    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "96c031cb-ffcf-435e-bfe6-3cc4ddf27295" (UID: "96c031cb-ffcf-435e-bfe6-3cc4ddf27295"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:08:01.321569 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.321542    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsss9\" (UniqueName: \"kubernetes.io/projected/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-kube-api-access-wsss9\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:08:01.321726 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.321571    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tls-certs\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:08:01.321726 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.321585    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-dshm\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:08:01.321726 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.321595    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c031cb-ffcf-435e-bfe6-3cc4ddf27295-tmp-dir\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:08:01.815675 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.815647    2576 generic.go:356] "Generic (PLEG): container finished" podID="96c031cb-ffcf-435e-bfe6-3cc4ddf27295" containerID="eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968" exitCode=0
Jul 22 19:08:01.815827 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.815685    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"
Jul 22 19:08:01.815827 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.815725    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z" event={"ID":"96c031cb-ffcf-435e-bfe6-3cc4ddf27295","Type":"ContainerDied","Data":"eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968"}
Jul 22 19:08:01.815827 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.815771    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z" event={"ID":"96c031cb-ffcf-435e-bfe6-3cc4ddf27295","Type":"ContainerDied","Data":"93bf7f48e6f7b390f294c3fd05f765ddc73186a6a5ee41414bb4400906546770"}
Jul 22 19:08:01.815827 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.815789    2576 scope.go:117] "RemoveContainer" containerID="eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968"
Jul 22 19:08:01.824400 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.824383    2576 scope.go:117] "RemoveContainer" containerID="eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968"
Jul 22 19:08:01.824681 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:08:01.824662    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968\": container with ID starting with eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968 not found: ID does not exist" containerID="eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968"
Jul 22 19:08:01.824744 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.824689    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968"} err="failed to get container status \"eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968\": rpc error: code = NotFound desc = could not find container \"eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968\": container with ID starting with eab40fb84ea394caefe62efc39c6833b93332161e6e78ede49c95dfbd039c968 not found: ID does not exist"
Jul 22 19:08:01.839677 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.839651    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"]
Jul 22 19:08:01.844803 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:01.844783    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-df776eca-kserve-64bb5dcb6c-wjh7z"]
Jul 22 19:08:03.687784 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:08:03.687753    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96c031cb-ffcf-435e-bfe6-3cc4ddf27295" path="/var/lib/kubelet/pods/96c031cb-ffcf-435e-bfe6-3cc4ddf27295/volumes"
Jul 22 19:11:38.992953 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:38.992875    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"]
Jul 22 19:11:38.993505 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:38.993284    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c031cb-ffcf-435e-bfe6-3cc4ddf27295" containerName="main"
Jul 22 19:11:38.993505 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:38.993297    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c031cb-ffcf-435e-bfe6-3cc4ddf27295" containerName="main"
Jul 22 19:11:38.993505 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:38.993382    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c031cb-ffcf-435e-bfe6-3cc4ddf27295" containerName="main"
Jul 22 19:11:38.996426 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:38.996407    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:38.999157 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:38.999136    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-3c49b710-serving-cert\""
Jul 22 19:11:39.010374 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.010354    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"]
Jul 22 19:11:39.111461 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.111427    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9519ba2a-89ab-430c-b764-77f18ce3e42b-openshift-service-ca\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.111461 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.111464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnswd\" (UniqueName: \"kubernetes.io/projected/9519ba2a-89ab-430c-b764-77f18ce3e42b-kube-api-access-fnswd\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.111674 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.111510    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9519ba2a-89ab-430c-b764-77f18ce3e42b-maas-api-tls\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.212645 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.212595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9519ba2a-89ab-430c-b764-77f18ce3e42b-openshift-service-ca\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.212788 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.212661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnswd\" (UniqueName: \"kubernetes.io/projected/9519ba2a-89ab-430c-b764-77f18ce3e42b-kube-api-access-fnswd\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.212788 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.212714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9519ba2a-89ab-430c-b764-77f18ce3e42b-maas-api-tls\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.212897 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:11:39.212879    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-serving-cert: secret "maas-api-e2e-ait-3c49b710-serving-cert" not found
Jul 22 19:11:39.212974 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:11:39.212963    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9519ba2a-89ab-430c-b764-77f18ce3e42b-maas-api-tls podName:9519ba2a-89ab-430c-b764-77f18ce3e42b nodeName:}" failed. No retries permitted until 2026-07-22 19:11:39.712941777 +0000 UTC m=+1900.603482689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9519ba2a-89ab-430c-b764-77f18ce3e42b-maas-api-tls") pod "maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh" (UID: "9519ba2a-89ab-430c-b764-77f18ce3e42b") : secret "maas-api-e2e-ait-3c49b710-serving-cert" not found
Jul 22 19:11:39.213326 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.213309    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9519ba2a-89ab-430c-b764-77f18ce3e42b-openshift-service-ca\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.229601 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.229578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnswd\" (UniqueName: \"kubernetes.io/projected/9519ba2a-89ab-430c-b764-77f18ce3e42b-kube-api-access-fnswd\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.718422 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.718392    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9519ba2a-89ab-430c-b764-77f18ce3e42b-maas-api-tls\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.720861 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.720836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9519ba2a-89ab-430c-b764-77f18ce3e42b-maas-api-tls\") pod \"maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:39.907515 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:39.907482    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:40.048781 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:40.048757    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"]
Jul 22 19:11:40.051751 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:11:40.051719    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9519ba2a_89ab_430c_b764_77f18ce3e42b.slice/crio-ff042fb83c8a9ce238140e56b9fba4d035aab234c9c1416ca1cb3f5d047e0dc6 WatchSource:0}: Error finding container ff042fb83c8a9ce238140e56b9fba4d035aab234c9c1416ca1cb3f5d047e0dc6: Status 404 returned error can't find the container with id ff042fb83c8a9ce238140e56b9fba4d035aab234c9c1416ca1cb3f5d047e0dc6
Jul 22 19:11:40.053052 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:40.053033    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 19:11:40.881970 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:40.881940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh" event={"ID":"9519ba2a-89ab-430c-b764-77f18ce3e42b","Type":"ContainerStarted","Data":"a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e"}
Jul 22 19:11:40.882159 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:40.881995    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:11:40.882159 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:40.882012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh" event={"ID":"9519ba2a-89ab-430c-b764-77f18ce3e42b","Type":"ContainerStarted","Data":"ff042fb83c8a9ce238140e56b9fba4d035aab234c9c1416ca1cb3f5d047e0dc6"}
Jul 22 19:11:40.903522 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:40.903462    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh" podStartSLOduration=2.342392598 podStartE2EDuration="2.903441268s" totalImagesPullingTime="561.04867ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:11:40.901540185 +0000 UTC m=+1901.792081117" watchObservedRunningTime="2026-07-22 19:11:40.903441268 +0000 UTC m=+1901.793982202"
Jul 22 19:11:46.892556 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:11:46.892524    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:12:14.987381 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:14.987333    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"]
Jul 22 19:12:14.987990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:14.987558    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh" podUID="9519ba2a-89ab-430c-b764-77f18ce3e42b" containerName="maas-api" containerID="cri-o://a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e" gracePeriod=30
Jul 22 19:12:15.241186 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.241126    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:12:15.319466 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.319435    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9519ba2a-89ab-430c-b764-77f18ce3e42b-openshift-service-ca\") pod \"9519ba2a-89ab-430c-b764-77f18ce3e42b\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") "
Jul 22 19:12:15.319466 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.319467    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9519ba2a-89ab-430c-b764-77f18ce3e42b-maas-api-tls\") pod \"9519ba2a-89ab-430c-b764-77f18ce3e42b\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") "
Jul 22 19:12:15.319680 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.319560    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnswd\" (UniqueName: \"kubernetes.io/projected/9519ba2a-89ab-430c-b764-77f18ce3e42b-kube-api-access-fnswd\") pod \"9519ba2a-89ab-430c-b764-77f18ce3e42b\" (UID: \"9519ba2a-89ab-430c-b764-77f18ce3e42b\") "
Jul 22 19:12:15.319808 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.319786    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9519ba2a-89ab-430c-b764-77f18ce3e42b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9519ba2a-89ab-430c-b764-77f18ce3e42b" (UID: "9519ba2a-89ab-430c-b764-77f18ce3e42b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:12:15.321780 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.321749    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9519ba2a-89ab-430c-b764-77f18ce3e42b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9519ba2a-89ab-430c-b764-77f18ce3e42b" (UID: "9519ba2a-89ab-430c-b764-77f18ce3e42b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:12:15.321889 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.321842    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9519ba2a-89ab-430c-b764-77f18ce3e42b-kube-api-access-fnswd" (OuterVolumeSpecName: "kube-api-access-fnswd") pod "9519ba2a-89ab-430c-b764-77f18ce3e42b" (UID: "9519ba2a-89ab-430c-b764-77f18ce3e42b"). InnerVolumeSpecName "kube-api-access-fnswd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:12:15.420979 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.420947    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnswd\" (UniqueName: \"kubernetes.io/projected/9519ba2a-89ab-430c-b764-77f18ce3e42b-kube-api-access-fnswd\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:12:15.420979 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.420973    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9519ba2a-89ab-430c-b764-77f18ce3e42b-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:12:15.420979 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:15.420985    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9519ba2a-89ab-430c-b764-77f18ce3e42b-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:12:16.049796 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:16.049725    2576 generic.go:356] "Generic (PLEG): container finished" podID="9519ba2a-89ab-430c-b764-77f18ce3e42b" containerID="a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e" exitCode=0
Jul 22 19:12:16.049796 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:16.049761    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh" event={"ID":"9519ba2a-89ab-430c-b764-77f18ce3e42b","Type":"ContainerDied","Data":"a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e"}
Jul 22 19:12:16.049796 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:16.049778    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"
Jul 22 19:12:16.050279 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:16.049799    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh" event={"ID":"9519ba2a-89ab-430c-b764-77f18ce3e42b","Type":"ContainerDied","Data":"ff042fb83c8a9ce238140e56b9fba4d035aab234c9c1416ca1cb3f5d047e0dc6"}
Jul 22 19:12:16.050279 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:16.049822    2576 scope.go:117] "RemoveContainer" containerID="a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e"
Jul 22 19:12:16.061174 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:16.061145    2576 scope.go:117] "RemoveContainer" containerID="a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e"
Jul 22 19:12:16.061487 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:12:16.061458    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e\": container with ID starting with a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e not found: ID does not exist" containerID="a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e"
Jul 22 19:12:16.061580 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:16.061499    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e"} err="failed to get container status \"a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e\": rpc error: code = NotFound desc = could not find container \"a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e\": container with ID starting with a72175cae6f9f865edd16f7b7b3e4f66771a39a668493b2b02c5d1bc9e672d7e not found: ID does not exist"
Jul 22 19:12:16.068334 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:16.068310    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"]
Jul 22 19:12:16.070470 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:16.070446    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3c49b710-55967f5cc9-bw9kh"]
Jul 22 19:12:17.688024 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:17.687986    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9519ba2a-89ab-430c-b764-77f18ce3e42b" path="/var/lib/kubelet/pods/9519ba2a-89ab-430c-b764-77f18ce3e42b/volumes"
Jul 22 19:12:23.288148 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.288112    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"]
Jul 22 19:12:23.288517 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.288470    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9519ba2a-89ab-430c-b764-77f18ce3e42b" containerName="maas-api"
Jul 22 19:12:23.288517 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.288480    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9519ba2a-89ab-430c-b764-77f18ce3e42b" containerName="maas-api"
Jul 22 19:12:23.288604 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.288551    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9519ba2a-89ab-430c-b764-77f18ce3e42b" containerName="maas-api"
Jul 22 19:12:23.293241 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.293213    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:23.295791 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.295766    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-666c748b-serving-cert\""
Jul 22 19:12:23.302952 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.302926    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"]
Jul 22 19:12:23.392206 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.392171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8vds\" (UniqueName: \"kubernetes.io/projected/e55a9038-7bbb-4e18-8372-54c5260d4bad-kube-api-access-l8vds\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:23.392527 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.392242    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55a9038-7bbb-4e18-8372-54c5260d4bad-openshift-service-ca\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:23.392527 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.392362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55a9038-7bbb-4e18-8372-54c5260d4bad-maas-api-tls\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:23.493162 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.493128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8vds\" (UniqueName: \"kubernetes.io/projected/e55a9038-7bbb-4e18-8372-54c5260d4bad-kube-api-access-l8vds\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:23.493299 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.493173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55a9038-7bbb-4e18-8372-54c5260d4bad-openshift-service-ca\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:23.493340 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.493298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55a9038-7bbb-4e18-8372-54c5260d4bad-maas-api-tls\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:23.493444 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:12:23.493428    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-serving-cert: secret "maas-api-e2e-migrate-666c748b-serving-cert" not found
Jul 22 19:12:23.493497 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:12:23.493488    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e55a9038-7bbb-4e18-8372-54c5260d4bad-maas-api-tls podName:e55a9038-7bbb-4e18-8372-54c5260d4bad nodeName:}" failed. No retries permitted until 2026-07-22 19:12:23.993471447 +0000 UTC m=+1944.884012360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e55a9038-7bbb-4e18-8372-54c5260d4bad-maas-api-tls") pod "maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg" (UID: "e55a9038-7bbb-4e18-8372-54c5260d4bad") : secret "maas-api-e2e-migrate-666c748b-serving-cert" not found
Jul 22 19:12:23.493774 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.493759    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55a9038-7bbb-4e18-8372-54c5260d4bad-openshift-service-ca\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:23.502380 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.502362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8vds\" (UniqueName: \"kubernetes.io/projected/e55a9038-7bbb-4e18-8372-54c5260d4bad-kube-api-access-l8vds\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:23.998596 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:23.998558    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55a9038-7bbb-4e18-8372-54c5260d4bad-maas-api-tls\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:24.000963 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:24.000945    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55a9038-7bbb-4e18-8372-54c5260d4bad-maas-api-tls\") pod \"maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:24.204882 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:24.204847    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:24.337865 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:24.337675    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"]
Jul 22 19:12:24.340737 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:12:24.340703    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode55a9038_7bbb_4e18_8372_54c5260d4bad.slice/crio-b057c832bb5cc8af727e43ab3962ac1b5ea4a69efaf8a693ea2e26847bdde15c WatchSource:0}: Error finding container b057c832bb5cc8af727e43ab3962ac1b5ea4a69efaf8a693ea2e26847bdde15c: Status 404 returned error can't find the container with id b057c832bb5cc8af727e43ab3962ac1b5ea4a69efaf8a693ea2e26847bdde15c
Jul 22 19:12:25.091523 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:25.091489    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg" event={"ID":"e55a9038-7bbb-4e18-8372-54c5260d4bad","Type":"ContainerStarted","Data":"20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418"}
Jul 22 19:12:25.091701 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:25.091543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg" event={"ID":"e55a9038-7bbb-4e18-8372-54c5260d4bad","Type":"ContainerStarted","Data":"b057c832bb5cc8af727e43ab3962ac1b5ea4a69efaf8a693ea2e26847bdde15c"}
Jul 22 19:12:25.091701 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:25.091572    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:25.110346 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:25.110303    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg" podStartSLOduration=1.5743217390000002 podStartE2EDuration="2.110288572s" totalImagesPullingTime="535.966833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:12:25.109469237 +0000 UTC m=+1946.000010171" watchObservedRunningTime="2026-07-22 19:12:25.110288572 +0000 UTC m=+1946.000829503"
Jul 22 19:12:31.101960 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:31.101933    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:12:48.270324 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:48.270292    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"]
Jul 22 19:12:48.273739 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:48.273722    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"
Jul 22 19:12:48.276604 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:48.276583    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5ttk\""
Jul 22 19:12:48.282796 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:48.282772    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"]
Jul 22 19:12:48.296944 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:48.296924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4751d96-0fd1-4732-b0c9-0c09d3e633d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-666c748b-h86p8\" (UID: \"e4751d96-0fd1-4732-b0c9-0c09d3e633d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"
Jul 22 19:12:48.398366 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:48.398334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4751d96-0fd1-4732-b0c9-0c09d3e633d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-666c748b-h86p8\" (UID: \"e4751d96-0fd1-4732-b0c9-0c09d3e633d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"
Jul 22 19:12:48.398953 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:48.398934    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4751d96-0fd1-4732-b0c9-0c09d3e633d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-666c748b-h86p8\" (UID: \"e4751d96-0fd1-4732-b0c9-0c09d3e633d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"
Jul 22 19:12:48.586120 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:48.586053    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"
Jul 22 19:12:48.718638 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:48.718602    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"]
Jul 22 19:12:48.720697 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:12:48.720669    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4751d96_0fd1_4732_b0c9_0c09d3e633d4.slice/crio-5c43a4049c1be0002a0ebcebf34bec560d975d22c084eb1f1da9f07d5d96758a WatchSource:0}: Error finding container 5c43a4049c1be0002a0ebcebf34bec560d975d22c084eb1f1da9f07d5d96758a: Status 404 returned error can't find the container with id 5c43a4049c1be0002a0ebcebf34bec560d975d22c084eb1f1da9f07d5d96758a
Jul 22 19:12:49.213237 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:49.213197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8" event={"ID":"e4751d96-0fd1-4732-b0c9-0c09d3e633d4","Type":"ContainerStarted","Data":"5c43a4049c1be0002a0ebcebf34bec560d975d22c084eb1f1da9f07d5d96758a"}
Jul 22 19:12:56.248264 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:56.248224    2576 generic.go:356] "Generic (PLEG): container finished" podID="e4751d96-0fd1-4732-b0c9-0c09d3e633d4" containerID="6c20a8f9af903c0cd69c89fc17ea7b13206aca45a9e67805812ac0e6e073fa8a" exitCode=0
Jul 22 19:12:56.248615 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:56.248311    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8" event={"ID":"e4751d96-0fd1-4732-b0c9-0c09d3e633d4","Type":"ContainerDied","Data":"6c20a8f9af903c0cd69c89fc17ea7b13206aca45a9e67805812ac0e6e073fa8a"}
Jul 22 19:12:57.385913 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:57.385887    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"
Jul 22 19:12:57.478292 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:57.478261    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4751d96-0fd1-4732-b0c9-0c09d3e633d4-maas-api-service-ca\") pod \"e4751d96-0fd1-4732-b0c9-0c09d3e633d4\" (UID: \"e4751d96-0fd1-4732-b0c9-0c09d3e633d4\") "
Jul 22 19:12:57.478598 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:57.478574    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4751d96-0fd1-4732-b0c9-0c09d3e633d4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4751d96-0fd1-4732-b0c9-0c09d3e633d4" (UID: "e4751d96-0fd1-4732-b0c9-0c09d3e633d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:12:57.579770 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:57.579689    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4751d96-0fd1-4732-b0c9-0c09d3e633d4-maas-api-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:12:58.259159 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:58.259128    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"
Jul 22 19:12:58.259326 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:58.259118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8" event={"ID":"e4751d96-0fd1-4732-b0c9-0c09d3e633d4","Type":"ContainerDied","Data":"5c43a4049c1be0002a0ebcebf34bec560d975d22c084eb1f1da9f07d5d96758a"}
Jul 22 19:12:58.259326 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:12:58.259275    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c43a4049c1be0002a0ebcebf34bec560d975d22c084eb1f1da9f07d5d96758a"
Jul 22 19:13:08.768021 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:08.767988    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"]
Jul 22 19:13:08.768456 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:08.768199    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg" podUID="e55a9038-7bbb-4e18-8372-54c5260d4bad" containerName="maas-api" containerID="cri-o://20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418" gracePeriod=30
Jul 22 19:13:09.024887 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.024832    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:13:09.172824 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.172793    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55a9038-7bbb-4e18-8372-54c5260d4bad-maas-api-tls\") pod \"e55a9038-7bbb-4e18-8372-54c5260d4bad\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") "
Jul 22 19:13:09.173000 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.172842    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8vds\" (UniqueName: \"kubernetes.io/projected/e55a9038-7bbb-4e18-8372-54c5260d4bad-kube-api-access-l8vds\") pod \"e55a9038-7bbb-4e18-8372-54c5260d4bad\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") "
Jul 22 19:13:09.173000 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.172867    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55a9038-7bbb-4e18-8372-54c5260d4bad-openshift-service-ca\") pod \"e55a9038-7bbb-4e18-8372-54c5260d4bad\" (UID: \"e55a9038-7bbb-4e18-8372-54c5260d4bad\") "
Jul 22 19:13:09.173241 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.173220    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e55a9038-7bbb-4e18-8372-54c5260d4bad-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e55a9038-7bbb-4e18-8372-54c5260d4bad" (UID: "e55a9038-7bbb-4e18-8372-54c5260d4bad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:13:09.174905 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.174881    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e55a9038-7bbb-4e18-8372-54c5260d4bad-kube-api-access-l8vds" (OuterVolumeSpecName: "kube-api-access-l8vds") pod "e55a9038-7bbb-4e18-8372-54c5260d4bad" (UID: "e55a9038-7bbb-4e18-8372-54c5260d4bad"). InnerVolumeSpecName "kube-api-access-l8vds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:13:09.174965 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.174904    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e55a9038-7bbb-4e18-8372-54c5260d4bad-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e55a9038-7bbb-4e18-8372-54c5260d4bad" (UID: "e55a9038-7bbb-4e18-8372-54c5260d4bad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:13:09.274483 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.274450    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55a9038-7bbb-4e18-8372-54c5260d4bad-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:13:09.274483 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.274479    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8vds\" (UniqueName: \"kubernetes.io/projected/e55a9038-7bbb-4e18-8372-54c5260d4bad-kube-api-access-l8vds\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:13:09.274677 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.274493    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55a9038-7bbb-4e18-8372-54c5260d4bad-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:13:09.311961 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.311891    2576 generic.go:356] "Generic (PLEG): container finished" podID="e55a9038-7bbb-4e18-8372-54c5260d4bad" containerID="20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418" exitCode=0
Jul 22 19:13:09.311961 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.311941    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"
Jul 22 19:13:09.312104 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.311973    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg" event={"ID":"e55a9038-7bbb-4e18-8372-54c5260d4bad","Type":"ContainerDied","Data":"20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418"}
Jul 22 19:13:09.312104 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.312032    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg" event={"ID":"e55a9038-7bbb-4e18-8372-54c5260d4bad","Type":"ContainerDied","Data":"b057c832bb5cc8af727e43ab3962ac1b5ea4a69efaf8a693ea2e26847bdde15c"}
Jul 22 19:13:09.312104 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.312059    2576 scope.go:117] "RemoveContainer" containerID="20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418"
Jul 22 19:13:09.322447 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.322316    2576 scope.go:117] "RemoveContainer" containerID="20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418"
Jul 22 19:13:09.324358 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:13:09.324331    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418\": container with ID starting with 20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418 not found: ID does not exist" containerID="20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418"
Jul 22 19:13:09.324468 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.324369    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418"} err="failed to get container status \"20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418\": rpc error: code = NotFound desc = could not find container \"20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418\": container with ID starting with 20223841d23d286bad26a8d13a67c88b6a1b59bf2f0371753b008d90c82f5418 not found: ID does not exist"
Jul 22 19:13:09.339226 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.339196    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"]
Jul 22 19:13:09.344705 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.344684    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-666c748b-55bd8bb8b5-fwxzg"]
Jul 22 19:13:09.689111 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:09.689070    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e55a9038-7bbb-4e18-8372-54c5260d4bad" path="/var/lib/kubelet/pods/e55a9038-7bbb-4e18-8372-54c5260d4bad/volumes"
Jul 22 19:13:13.345540 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:13.345505    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"]
Jul 22 19:13:13.349384 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:13.349360    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-666c748b-h86p8"]
Jul 22 19:13:13.688253 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:13.688222    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4751d96-0fd1-4732-b0c9-0c09d3e633d4" path="/var/lib/kubelet/pods/e4751d96-0fd1-4732-b0c9-0c09d3e633d4/volumes"
Jul 22 19:13:16.685492 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.685454    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"]
Jul 22 19:13:16.686025 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.686005    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e55a9038-7bbb-4e18-8372-54c5260d4bad" containerName="maas-api"
Jul 22 19:13:16.686109 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.686029    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e55a9038-7bbb-4e18-8372-54c5260d4bad" containerName="maas-api"
Jul 22 19:13:16.686109 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.686048    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4751d96-0fd1-4732-b0c9-0c09d3e633d4" containerName="revoke-keys"
Jul 22 19:13:16.686109 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.686057    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4751d96-0fd1-4732-b0c9-0c09d3e633d4" containerName="revoke-keys"
Jul 22 19:13:16.686259 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.686161    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4751d96-0fd1-4732-b0c9-0c09d3e633d4" containerName="revoke-keys"
Jul 22 19:13:16.686259 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.686178    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e55a9038-7bbb-4e18-8372-54c5260d4bad" containerName="maas-api"
Jul 22 19:13:16.690489 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.690467    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:16.693115 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.693097    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9f921dcc-serving-cert\""
Jul 22 19:13:16.699027 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.699005    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"]
Jul 22 19:13:16.839907 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.839874    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322b6b92-27d8-477b-9007-1529e0788331-openshift-service-ca\") pod \"maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:16.839907 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.839912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvh8n\" (UniqueName: \"kubernetes.io/projected/322b6b92-27d8-477b-9007-1529e0788331-kube-api-access-zvh8n\") pod \"maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:16.840119 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.839973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322b6b92-27d8-477b-9007-1529e0788331-maas-api-tls\") pod \"maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:16.940684 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.940563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322b6b92-27d8-477b-9007-1529e0788331-openshift-service-ca\") pod \"maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:16.940684 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.940616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvh8n\" (UniqueName: \"kubernetes.io/projected/322b6b92-27d8-477b-9007-1529e0788331-kube-api-access-zvh8n\") pod \"maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:16.940896 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.940698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322b6b92-27d8-477b-9007-1529e0788331-maas-api-tls\") pod \"maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:16.941189 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.941159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322b6b92-27d8-477b-9007-1529e0788331-openshift-service-ca\") pod \"maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:16.943162 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.943139    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322b6b92-27d8-477b-9007-1529e0788331-maas-api-tls\") pod \"maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:16.952212 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:16.952187    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvh8n\" (UniqueName: \"kubernetes.io/projected/322b6b92-27d8-477b-9007-1529e0788331-kube-api-access-zvh8n\") pod \"maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:17.002146 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:17.002122    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:17.138326 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:17.137698    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"]
Jul 22 19:13:17.141437 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:13:17.141402    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod322b6b92_27d8_477b_9007_1529e0788331.slice/crio-42bd30472e20d742831651f142f25ef3cb708ee6d02b3cfd0ccdcf52731a054f WatchSource:0}: Error finding container 42bd30472e20d742831651f142f25ef3cb708ee6d02b3cfd0ccdcf52731a054f: Status 404 returned error can't find the container with id 42bd30472e20d742831651f142f25ef3cb708ee6d02b3cfd0ccdcf52731a054f
Jul 22 19:13:17.353522 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:17.353433    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg" event={"ID":"322b6b92-27d8-477b-9007-1529e0788331","Type":"ContainerStarted","Data":"42bd30472e20d742831651f142f25ef3cb708ee6d02b3cfd0ccdcf52731a054f"}
Jul 22 19:13:18.360923 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:18.360883    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg" event={"ID":"322b6b92-27d8-477b-9007-1529e0788331","Type":"ContainerStarted","Data":"1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b"}
Jul 22 19:13:18.361305 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:18.361011    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:18.384111 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:18.384059    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg" podStartSLOduration=1.89191752 podStartE2EDuration="2.384041455s" totalImagesPullingTime="492.123935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:13:18.383649247 +0000 UTC m=+1999.274190178" watchObservedRunningTime="2026-07-22 19:13:18.384041455 +0000 UTC m=+1999.274582391"
Jul 22 19:13:24.371032 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:24.371002    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:42.343067 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:42.343028    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"]
Jul 22 19:13:42.346460 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:42.346440    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"
Jul 22 19:13:42.349058 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:42.349036    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5ttk\""
Jul 22 19:13:42.353926 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:42.353523    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"]
Jul 22 19:13:42.458258 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:42.458224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69269bd1-be19-46e4-94a9-2a4e6624fd6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6\" (UID: \"69269bd1-be19-46e4-94a9-2a4e6624fd6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"
Jul 22 19:13:42.559440 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:42.559409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69269bd1-be19-46e4-94a9-2a4e6624fd6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6\" (UID: \"69269bd1-be19-46e4-94a9-2a4e6624fd6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"
Jul 22 19:13:42.560075 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:42.560053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69269bd1-be19-46e4-94a9-2a4e6624fd6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6\" (UID: \"69269bd1-be19-46e4-94a9-2a4e6624fd6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"
Jul 22 19:13:42.660039 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:42.660011    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"
Jul 22 19:13:42.799403 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:42.799360    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"]
Jul 22 19:13:42.806670 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:13:42.801900    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69269bd1_be19_46e4_94a9_2a4e6624fd6c.slice/crio-767b15e87dc88c036fe4e284655da38a104318eaff3d5cd5bc8d0dbe2f6294d7 WatchSource:0}: Error finding container 767b15e87dc88c036fe4e284655da38a104318eaff3d5cd5bc8d0dbe2f6294d7: Status 404 returned error can't find the container with id 767b15e87dc88c036fe4e284655da38a104318eaff3d5cd5bc8d0dbe2f6294d7
Jul 22 19:13:43.482787 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:43.482746    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6" event={"ID":"69269bd1-be19-46e4-94a9-2a4e6624fd6c","Type":"ContainerStarted","Data":"f76c4c86f4354b2505048502765e98682021ddce388abdd8efad2c4f95dfdc17"}
Jul 22 19:13:43.483213 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:43.482837    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6" event={"ID":"69269bd1-be19-46e4-94a9-2a4e6624fd6c","Type":"ContainerStarted","Data":"767b15e87dc88c036fe4e284655da38a104318eaff3d5cd5bc8d0dbe2f6294d7"}
Jul 22 19:13:43.500939 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:43.500892    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6" podStartSLOduration=1.500879461 podStartE2EDuration="1.500879461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:13:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:13:43.499095872 +0000 UTC m=+2024.389636804" watchObservedRunningTime="2026-07-22 19:13:43.500879461 +0000 UTC m=+2024.391420445"
Jul 22 19:13:44.489048 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:44.489014    2576 generic.go:356] "Generic (PLEG): container finished" podID="69269bd1-be19-46e4-94a9-2a4e6624fd6c" containerID="f76c4c86f4354b2505048502765e98682021ddce388abdd8efad2c4f95dfdc17" exitCode=0
Jul 22 19:13:44.489457 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:44.489100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6" event={"ID":"69269bd1-be19-46e4-94a9-2a4e6624fd6c","Type":"ContainerDied","Data":"f76c4c86f4354b2505048502765e98682021ddce388abdd8efad2c4f95dfdc17"}
Jul 22 19:13:45.623778 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:45.623753    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"
Jul 22 19:13:45.685979 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:45.685945    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69269bd1-be19-46e4-94a9-2a4e6624fd6c-maas-api-service-ca\") pod \"69269bd1-be19-46e4-94a9-2a4e6624fd6c\" (UID: \"69269bd1-be19-46e4-94a9-2a4e6624fd6c\") "
Jul 22 19:13:45.686295 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:45.686270    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69269bd1-be19-46e4-94a9-2a4e6624fd6c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "69269bd1-be19-46e4-94a9-2a4e6624fd6c" (UID: "69269bd1-be19-46e4-94a9-2a4e6624fd6c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:13:45.787105 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:45.787045    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69269bd1-be19-46e4-94a9-2a4e6624fd6c-maas-api-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:13:46.501083 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:46.501002    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"
Jul 22 19:13:46.501083 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:46.501038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6" event={"ID":"69269bd1-be19-46e4-94a9-2a4e6624fd6c","Type":"ContainerDied","Data":"767b15e87dc88c036fe4e284655da38a104318eaff3d5cd5bc8d0dbe2f6294d7"}
Jul 22 19:13:46.501083 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:46.501084    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="767b15e87dc88c036fe4e284655da38a104318eaff3d5cd5bc8d0dbe2f6294d7"
Jul 22 19:13:53.147391 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.147359    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"]
Jul 22 19:13:53.147817 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.147580    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg" podUID="322b6b92-27d8-477b-9007-1529e0788331" containerName="maas-api" containerID="cri-o://1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b" gracePeriod=30
Jul 22 19:13:53.397758 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.397697    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:53.444153 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.444118    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322b6b92-27d8-477b-9007-1529e0788331-openshift-service-ca\") pod \"322b6b92-27d8-477b-9007-1529e0788331\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") "
Jul 22 19:13:53.444341 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.444229    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322b6b92-27d8-477b-9007-1529e0788331-maas-api-tls\") pod \"322b6b92-27d8-477b-9007-1529e0788331\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") "
Jul 22 19:13:53.444341 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.444263    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvh8n\" (UniqueName: \"kubernetes.io/projected/322b6b92-27d8-477b-9007-1529e0788331-kube-api-access-zvh8n\") pod \"322b6b92-27d8-477b-9007-1529e0788331\" (UID: \"322b6b92-27d8-477b-9007-1529e0788331\") "
Jul 22 19:13:53.444484 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.444458    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/322b6b92-27d8-477b-9007-1529e0788331-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "322b6b92-27d8-477b-9007-1529e0788331" (UID: "322b6b92-27d8-477b-9007-1529e0788331"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:13:53.444822 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.444797    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322b6b92-27d8-477b-9007-1529e0788331-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:13:53.446254 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.446223    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/322b6b92-27d8-477b-9007-1529e0788331-kube-api-access-zvh8n" (OuterVolumeSpecName: "kube-api-access-zvh8n") pod "322b6b92-27d8-477b-9007-1529e0788331" (UID: "322b6b92-27d8-477b-9007-1529e0788331"). InnerVolumeSpecName "kube-api-access-zvh8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:13:53.446373 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.446349    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/322b6b92-27d8-477b-9007-1529e0788331-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "322b6b92-27d8-477b-9007-1529e0788331" (UID: "322b6b92-27d8-477b-9007-1529e0788331"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:13:53.537493 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.537463    2576 generic.go:356] "Generic (PLEG): container finished" podID="322b6b92-27d8-477b-9007-1529e0788331" containerID="1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b" exitCode=0
Jul 22 19:13:53.537618 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.537524    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"
Jul 22 19:13:53.537618 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.537538    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg" event={"ID":"322b6b92-27d8-477b-9007-1529e0788331","Type":"ContainerDied","Data":"1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b"}
Jul 22 19:13:53.537618 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.537593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg" event={"ID":"322b6b92-27d8-477b-9007-1529e0788331","Type":"ContainerDied","Data":"42bd30472e20d742831651f142f25ef3cb708ee6d02b3cfd0ccdcf52731a054f"}
Jul 22 19:13:53.537618 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.537617    2576 scope.go:117] "RemoveContainer" containerID="1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b"
Jul 22 19:13:53.545309 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.545287    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322b6b92-27d8-477b-9007-1529e0788331-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:13:53.545309 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.545310    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvh8n\" (UniqueName: \"kubernetes.io/projected/322b6b92-27d8-477b-9007-1529e0788331-kube-api-access-zvh8n\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:13:53.546820 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.546800    2576 scope.go:117] "RemoveContainer" containerID="1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b"
Jul 22 19:13:53.547187 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:13:53.547161    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b\": container with ID starting with 1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b not found: ID does not exist" containerID="1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b"
Jul 22 19:13:53.547264 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.547196    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b"} err="failed to get container status \"1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b\": rpc error: code = NotFound desc = could not find container \"1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b\": container with ID starting with 1b3a899c2b274df04ae3af5b354b732ffb34d602242f5498ddfd7a928161471b not found: ID does not exist"
Jul 22 19:13:53.560411 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.560391    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"]
Jul 22 19:13:53.562063 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.562044    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f921dcc-85fb7fb79-csjmg"]
Jul 22 19:13:53.688659 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:53.688575    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="322b6b92-27d8-477b-9007-1529e0788331" path="/var/lib/kubelet/pods/322b6b92-27d8-477b-9007-1529e0788331/volumes"
Jul 22 19:13:57.420375 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:57.420341    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"]
Jul 22 19:13:57.422819 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:57.422794    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9f921dcc-kwfg6"]
Jul 22 19:13:57.687964 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:13:57.687877    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69269bd1-be19-46e4-94a9-2a4e6624fd6c" path="/var/lib/kubelet/pods/69269bd1-be19-46e4-94a9-2a4e6624fd6c/volumes"
Jul 22 19:14:01.788529 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.788501    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"]
Jul 22 19:14:01.788954 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.788908    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69269bd1-be19-46e4-94a9-2a4e6624fd6c" containerName="revoke-keys"
Jul 22 19:14:01.788954 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.788923    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="69269bd1-be19-46e4-94a9-2a4e6624fd6c" containerName="revoke-keys"
Jul 22 19:14:01.788954 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.788943    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="322b6b92-27d8-477b-9007-1529e0788331" containerName="maas-api"
Jul 22 19:14:01.788954 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.788949    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="322b6b92-27d8-477b-9007-1529e0788331" containerName="maas-api"
Jul 22 19:14:01.789084 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.789010    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="322b6b92-27d8-477b-9007-1529e0788331" containerName="maas-api"
Jul 22 19:14:01.789084 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.789024    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="69269bd1-be19-46e4-94a9-2a4e6624fd6c" containerName="revoke-keys"
Jul 22 19:14:01.791932 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.791917    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:01.794790 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.794768    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-908afcb9-serving-cert\""
Jul 22 19:14:01.802667 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.802623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"]
Jul 22 19:14:01.929819 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.929784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-maas-api-tls\") pod \"maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:01.930164 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.930139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-openshift-service-ca\") pod \"maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:01.930343 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:01.930321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chvbb\" (UniqueName: \"kubernetes.io/projected/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-kube-api-access-chvbb\") pod \"maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:02.031127 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:02.031096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chvbb\" (UniqueName: \"kubernetes.io/projected/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-kube-api-access-chvbb\") pod \"maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:02.031301 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:02.031195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-maas-api-tls\") pod \"maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:02.031301 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:02.031237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-openshift-service-ca\") pod \"maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:02.032079 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:02.032053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-openshift-service-ca\") pod \"maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:02.034038 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:02.034014    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-maas-api-tls\") pod \"maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:02.041832 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:02.041786    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chvbb\" (UniqueName: \"kubernetes.io/projected/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-kube-api-access-chvbb\") pod \"maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:02.101674 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:02.101648    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:02.454956 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:02.454930    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"]
Jul 22 19:14:02.456945 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:14:02.456916    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4c6593_5b0f_4b11_bfff_c5098d3d206b.slice/crio-966b29808a086a9544fca729190649f807529009e8700945316c378caec8a0df WatchSource:0}: Error finding container 966b29808a086a9544fca729190649f807529009e8700945316c378caec8a0df: Status 404 returned error can't find the container with id 966b29808a086a9544fca729190649f807529009e8700945316c378caec8a0df
Jul 22 19:14:02.583666 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:02.583607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv" event={"ID":"aa4c6593-5b0f-4b11-bfff-c5098d3d206b","Type":"ContainerStarted","Data":"966b29808a086a9544fca729190649f807529009e8700945316c378caec8a0df"}
Jul 22 19:14:03.589285 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:03.589248    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv" event={"ID":"aa4c6593-5b0f-4b11-bfff-c5098d3d206b","Type":"ContainerStarted","Data":"a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d"}
Jul 22 19:14:03.589685 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:03.589391    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:03.607306 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:03.607259    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv" podStartSLOduration=2.273288359 podStartE2EDuration="2.607244257s" totalImagesPullingTime="333.955898ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:14:03.60522904 +0000 UTC m=+2044.495769972" watchObservedRunningTime="2026-07-22 19:14:03.607244257 +0000 UTC m=+2044.497785186"
Jul 22 19:14:09.599974 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:09.599948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:26.626370 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:26.626292    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"]
Jul 22 19:14:26.629879 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:26.629861    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"
Jul 22 19:14:26.632817 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:26.632792    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5ttk\""
Jul 22 19:14:26.639276 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:26.639250    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"]
Jul 22 19:14:26.644735 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:26.644705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd690690-8faf-4fd3-b6fb-7cca9b3d5be0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc\" (UID: \"fd690690-8faf-4fd3-b6fb-7cca9b3d5be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"
Jul 22 19:14:26.746012 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:26.745979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd690690-8faf-4fd3-b6fb-7cca9b3d5be0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc\" (UID: \"fd690690-8faf-4fd3-b6fb-7cca9b3d5be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"
Jul 22 19:14:26.746551 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:26.746533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd690690-8faf-4fd3-b6fb-7cca9b3d5be0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc\" (UID: \"fd690690-8faf-4fd3-b6fb-7cca9b3d5be0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"
Jul 22 19:14:26.943138 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:26.943067    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"
Jul 22 19:14:27.075370 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:27.075346    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"]
Jul 22 19:14:27.078166 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:14:27.078138    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd690690_8faf_4fd3_b6fb_7cca9b3d5be0.slice/crio-d6a4c9811a7d27f94ec2d5c0a67619f71430247694547b692882bdaa1a1e6d57 WatchSource:0}: Error finding container d6a4c9811a7d27f94ec2d5c0a67619f71430247694547b692882bdaa1a1e6d57: Status 404 returned error can't find the container with id d6a4c9811a7d27f94ec2d5c0a67619f71430247694547b692882bdaa1a1e6d57
Jul 22 19:14:27.706959 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:27.706925    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc" event={"ID":"fd690690-8faf-4fd3-b6fb-7cca9b3d5be0","Type":"ContainerStarted","Data":"ceeb106b07be2739cc352b04b06920358810956c77b9115ce4402fa33db68b19"}
Jul 22 19:14:27.707322 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:27.706972    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc" event={"ID":"fd690690-8faf-4fd3-b6fb-7cca9b3d5be0","Type":"ContainerStarted","Data":"d6a4c9811a7d27f94ec2d5c0a67619f71430247694547b692882bdaa1a1e6d57"}
Jul 22 19:14:27.726107 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:27.726062    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc" podStartSLOduration=1.726048636 podStartE2EDuration="1.726048636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:14:27.724147294 +0000 UTC m=+2068.614688226" watchObservedRunningTime="2026-07-22 19:14:27.726048636 +0000 UTC m=+2068.616589567"
Jul 22 19:14:28.713296 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:28.713258    2576 generic.go:356] "Generic (PLEG): container finished" podID="fd690690-8faf-4fd3-b6fb-7cca9b3d5be0" containerID="ceeb106b07be2739cc352b04b06920358810956c77b9115ce4402fa33db68b19" exitCode=0
Jul 22 19:14:28.713684 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:28.713340    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc" event={"ID":"fd690690-8faf-4fd3-b6fb-7cca9b3d5be0","Type":"ContainerDied","Data":"ceeb106b07be2739cc352b04b06920358810956c77b9115ce4402fa33db68b19"}
Jul 22 19:14:29.856602 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:29.856576    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"
Jul 22 19:14:29.973769 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:29.973701    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd690690-8faf-4fd3-b6fb-7cca9b3d5be0-maas-api-service-ca\") pod \"fd690690-8faf-4fd3-b6fb-7cca9b3d5be0\" (UID: \"fd690690-8faf-4fd3-b6fb-7cca9b3d5be0\") "
Jul 22 19:14:29.974019 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:29.973996    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd690690-8faf-4fd3-b6fb-7cca9b3d5be0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd690690-8faf-4fd3-b6fb-7cca9b3d5be0" (UID: "fd690690-8faf-4fd3-b6fb-7cca9b3d5be0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:14:30.075296 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:30.075260    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd690690-8faf-4fd3-b6fb-7cca9b3d5be0-maas-api-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:14:30.724099 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:30.724066    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"
Jul 22 19:14:30.724276 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:30.724098    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc" event={"ID":"fd690690-8faf-4fd3-b6fb-7cca9b3d5be0","Type":"ContainerDied","Data":"d6a4c9811a7d27f94ec2d5c0a67619f71430247694547b692882bdaa1a1e6d57"}
Jul 22 19:14:30.724276 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:30.724147    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6a4c9811a7d27f94ec2d5c0a67619f71430247694547b692882bdaa1a1e6d57"
Jul 22 19:14:37.090799 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.090758    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"]
Jul 22 19:14:37.091389 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.090971    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv" podUID="aa4c6593-5b0f-4b11-bfff-c5098d3d206b" containerName="maas-api" containerID="cri-o://a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d" gracePeriod=30
Jul 22 19:14:37.340603 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.340573    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:37.437697 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.437666    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-openshift-service-ca\") pod \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") "
Jul 22 19:14:37.437888 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.437716    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chvbb\" (UniqueName: \"kubernetes.io/projected/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-kube-api-access-chvbb\") pod \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") "
Jul 22 19:14:37.437888 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.437760    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-maas-api-tls\") pod \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\" (UID: \"aa4c6593-5b0f-4b11-bfff-c5098d3d206b\") "
Jul 22 19:14:37.438334 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.438308    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa4c6593-5b0f-4b11-bfff-c5098d3d206b" (UID: "aa4c6593-5b0f-4b11-bfff-c5098d3d206b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:14:37.439971 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.439945    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-kube-api-access-chvbb" (OuterVolumeSpecName: "kube-api-access-chvbb") pod "aa4c6593-5b0f-4b11-bfff-c5098d3d206b" (UID: "aa4c6593-5b0f-4b11-bfff-c5098d3d206b"). InnerVolumeSpecName "kube-api-access-chvbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:14:37.440207 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.440192    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa4c6593-5b0f-4b11-bfff-c5098d3d206b" (UID: "aa4c6593-5b0f-4b11-bfff-c5098d3d206b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:14:37.538602 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.538568    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chvbb\" (UniqueName: \"kubernetes.io/projected/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-kube-api-access-chvbb\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:14:37.538602 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.538598    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:14:37.538602 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.538609    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4c6593-5b0f-4b11-bfff-c5098d3d206b-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:14:37.757539 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.757510    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa4c6593-5b0f-4b11-bfff-c5098d3d206b" containerID="a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d" exitCode=0
Jul 22 19:14:37.757706 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.757557    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"
Jul 22 19:14:37.757706 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.757592    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv" event={"ID":"aa4c6593-5b0f-4b11-bfff-c5098d3d206b","Type":"ContainerDied","Data":"a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d"}
Jul 22 19:14:37.757706 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.757672    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv" event={"ID":"aa4c6593-5b0f-4b11-bfff-c5098d3d206b","Type":"ContainerDied","Data":"966b29808a086a9544fca729190649f807529009e8700945316c378caec8a0df"}
Jul 22 19:14:37.757706 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.757704    2576 scope.go:117] "RemoveContainer" containerID="a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d"
Jul 22 19:14:37.766570 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.766555    2576 scope.go:117] "RemoveContainer" containerID="a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d"
Jul 22 19:14:37.766839 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:14:37.766822    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d\": container with ID starting with a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d not found: ID does not exist" containerID="a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d"
Jul 22 19:14:37.766890 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.766846    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d"} err="failed to get container status \"a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d\": rpc error: code = NotFound desc = could not find container \"a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d\": container with ID starting with a693f0449fe0c32a5e3cb59fc2d06e6f735d2ad866541289a20c6a78117c9c1d not found: ID does not exist"
Jul 22 19:14:37.775581 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.775562    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"]
Jul 22 19:14:37.781121 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:37.781096    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-908afcb9-bd98ff789-9cgpv"]
Jul 22 19:14:39.688249 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:39.688221    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa4c6593-5b0f-4b11-bfff-c5098d3d206b" path="/var/lib/kubelet/pods/aa4c6593-5b0f-4b11-bfff-c5098d3d206b/volumes"
Jul 22 19:14:41.700934 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:41.700902    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"]
Jul 22 19:14:41.703161 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:41.703139    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-908afcb9-qxnnc"]
Jul 22 19:14:43.689175 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:14:43.689145    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd690690-8faf-4fd3-b6fb-7cca9b3d5be0" path="/var/lib/kubelet/pods/fd690690-8faf-4fd3-b6fb-7cca9b3d5be0/volumes"
Jul 22 19:15:00.132593 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.132560    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5"]
Jul 22 19:15:00.133104 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.133085    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4c6593-5b0f-4b11-bfff-c5098d3d206b" containerName="maas-api"
Jul 22 19:15:00.133194 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.133107    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4c6593-5b0f-4b11-bfff-c5098d3d206b" containerName="maas-api"
Jul 22 19:15:00.133194 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.133125    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd690690-8faf-4fd3-b6fb-7cca9b3d5be0" containerName="revoke-keys"
Jul 22 19:15:00.133194 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.133134    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd690690-8faf-4fd3-b6fb-7cca9b3d5be0" containerName="revoke-keys"
Jul 22 19:15:00.133296 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.133224    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4c6593-5b0f-4b11-bfff-c5098d3d206b" containerName="maas-api"
Jul 22 19:15:00.133296 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.133237    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd690690-8faf-4fd3-b6fb-7cca9b3d5be0" containerName="revoke-keys"
Jul 22 19:15:00.138174 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.138158    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5"
Jul 22 19:15:00.141016 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.140994    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5ttk\""
Jul 22 19:15:00.144990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.144946    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5"]
Jul 22 19:15:00.150785 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.150765    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5"
Jul 22 19:15:00.508978 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.508949    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5"]
Jul 22 19:15:00.510446 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:15:00.510411    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0096aa46_29d2_4595_acbb_95b92c1cf992.slice/crio-f6f7ba04d45601e60146a734fb0b40ac2f05e3a2b803690cb72554bb2c137f1e WatchSource:0}: Error finding container f6f7ba04d45601e60146a734fb0b40ac2f05e3a2b803690cb72554bb2c137f1e: Status 404 returned error can't find the container with id f6f7ba04d45601e60146a734fb0b40ac2f05e3a2b803690cb72554bb2c137f1e
Jul 22 19:15:00.868772 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.868740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5" event={"ID":"0096aa46-29d2-4595-acbb-95b92c1cf992","Type":"ContainerStarted","Data":"e85fc011d99a97908d349d215fdbb83cf20e3fa935db5f499b8adc3bd7169d17"}
Jul 22 19:15:00.868895 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.868804    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5" event={"ID":"0096aa46-29d2-4595-acbb-95b92c1cf992","Type":"ContainerStarted","Data":"f6f7ba04d45601e60146a734fb0b40ac2f05e3a2b803690cb72554bb2c137f1e"}
Jul 22 19:15:00.883932 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:00.883883    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5" podStartSLOduration=0.883864967 podStartE2EDuration="883.864967ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:15:00.882091634 +0000 UTC m=+2101.772632562" watchObservedRunningTime="2026-07-22 19:15:00.883864967 +0000 UTC m=+2101.774405900"
Jul 22 19:15:01.875133 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:01.875093    2576 generic.go:356] "Generic (PLEG): container finished" podID="0096aa46-29d2-4595-acbb-95b92c1cf992" containerID="e85fc011d99a97908d349d215fdbb83cf20e3fa935db5f499b8adc3bd7169d17" exitCode=0
Jul 22 19:15:01.875502 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:01.875193    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5" event={"ID":"0096aa46-29d2-4595-acbb-95b92c1cf992","Type":"ContainerDied","Data":"e85fc011d99a97908d349d215fdbb83cf20e3fa935db5f499b8adc3bd7169d17"}
Jul 22 19:15:03.027936 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:03.027909    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5"
Jul 22 19:15:03.888008 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:03.887983    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5"
Jul 22 19:15:03.888008 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:03.887993    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-b9ql5" event={"ID":"0096aa46-29d2-4595-acbb-95b92c1cf992","Type":"ContainerDied","Data":"f6f7ba04d45601e60146a734fb0b40ac2f05e3a2b803690cb72554bb2c137f1e"}
Jul 22 19:15:03.888231 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:15:03.888039    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6f7ba04d45601e60146a734fb0b40ac2f05e3a2b803690cb72554bb2c137f1e"
Jul 22 19:16:24.291857 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.291668    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"]
Jul 22 19:16:24.292983 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.292958    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0096aa46-29d2-4595-acbb-95b92c1cf992" containerName="cleanup"
Jul 22 19:16:24.293139 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.293116    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0096aa46-29d2-4595-acbb-95b92c1cf992" containerName="cleanup"
Jul 22 19:16:24.293285 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.293271    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0096aa46-29d2-4595-acbb-95b92c1cf992" containerName="cleanup"
Jul 22 19:16:24.296404 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.296384    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.299364 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.299339    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3579e215-serving-cert\""
Jul 22 19:16:24.306879 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.306800    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"]
Jul 22 19:16:24.360360 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.360309    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mtf6\" (UniqueName: \"kubernetes.io/projected/70960298-5cd8-46a5-a6ce-1ff91e5dc969-kube-api-access-8mtf6\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.360549 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.360415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70960298-5cd8-46a5-a6ce-1ff91e5dc969-maas-api-tls\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.360549 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.360512    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70960298-5cd8-46a5-a6ce-1ff91e5dc969-openshift-service-ca\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.461894 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.461857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70960298-5cd8-46a5-a6ce-1ff91e5dc969-maas-api-tls\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.462050 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.461962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70960298-5cd8-46a5-a6ce-1ff91e5dc969-openshift-service-ca\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.462050 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.461988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mtf6\" (UniqueName: \"kubernetes.io/projected/70960298-5cd8-46a5-a6ce-1ff91e5dc969-kube-api-access-8mtf6\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.462050 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:16:24.462018    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-serving-cert: secret "maas-api-e2e-mt-3579e215-serving-cert" not found
Jul 22 19:16:24.462159 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:16:24.462116    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70960298-5cd8-46a5-a6ce-1ff91e5dc969-maas-api-tls podName:70960298-5cd8-46a5-a6ce-1ff91e5dc969 nodeName:}" failed. No retries permitted until 2026-07-22 19:16:24.962092229 +0000 UTC m=+2185.852633139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70960298-5cd8-46a5-a6ce-1ff91e5dc969-maas-api-tls") pod "maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd" (UID: "70960298-5cd8-46a5-a6ce-1ff91e5dc969") : secret "maas-api-e2e-mt-3579e215-serving-cert" not found
Jul 22 19:16:24.462622 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.462605    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70960298-5cd8-46a5-a6ce-1ff91e5dc969-openshift-service-ca\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.473945 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.473915    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mtf6\" (UniqueName: \"kubernetes.io/projected/70960298-5cd8-46a5-a6ce-1ff91e5dc969-kube-api-access-8mtf6\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.967454 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.967419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70960298-5cd8-46a5-a6ce-1ff91e5dc969-maas-api-tls\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:24.973661 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:24.970486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70960298-5cd8-46a5-a6ce-1ff91e5dc969-maas-api-tls\") pod \"maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:25.208994 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:25.208957    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:25.380474 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:25.380445    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"]
Jul 22 19:16:25.382989 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:16:25.382960    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70960298_5cd8_46a5_a6ce_1ff91e5dc969.slice/crio-208f5afa00c2c6464a665edd17e1eeaeedf22e0d2f9cbb9ccf4e4b7a2f181667 WatchSource:0}: Error finding container 208f5afa00c2c6464a665edd17e1eeaeedf22e0d2f9cbb9ccf4e4b7a2f181667: Status 404 returned error can't find the container with id 208f5afa00c2c6464a665edd17e1eeaeedf22e0d2f9cbb9ccf4e4b7a2f181667
Jul 22 19:16:26.312416 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:26.312376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd" event={"ID":"70960298-5cd8-46a5-a6ce-1ff91e5dc969","Type":"ContainerStarted","Data":"208f5afa00c2c6464a665edd17e1eeaeedf22e0d2f9cbb9ccf4e4b7a2f181667"}
Jul 22 19:16:27.318603 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:27.318561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd" event={"ID":"70960298-5cd8-46a5-a6ce-1ff91e5dc969","Type":"ContainerStarted","Data":"d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df"}
Jul 22 19:16:27.319190 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:27.318654    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:27.362821 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:27.362754    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd" podStartSLOduration=2.36070471 podStartE2EDuration="3.362738538s" totalImagesPullingTime="1.002033828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:16:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:16:27.360550272 +0000 UTC m=+2188.251091202" watchObservedRunningTime="2026-07-22 19:16:27.362738538 +0000 UTC m=+2188.253279470"
Jul 22 19:16:33.329693 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:33.329665    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:35.953009 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:35.952975    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"]
Jul 22 19:16:35.955945 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:35.955928    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"
Jul 22 19:16:35.958944 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:35.958925    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5ttk\""
Jul 22 19:16:35.964546 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:35.964525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"]
Jul 22 19:16:36.066133 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:36.066105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c37f7df-2204-4474-a99d-5afb968b5279-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3579e215-7tkhv\" (UID: \"9c37f7df-2204-4474-a99d-5afb968b5279\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"
Jul 22 19:16:36.166842 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:36.166804    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c37f7df-2204-4474-a99d-5afb968b5279-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3579e215-7tkhv\" (UID: \"9c37f7df-2204-4474-a99d-5afb968b5279\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"
Jul 22 19:16:36.167499 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:36.167471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c37f7df-2204-4474-a99d-5afb968b5279-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3579e215-7tkhv\" (UID: \"9c37f7df-2204-4474-a99d-5afb968b5279\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"
Jul 22 19:16:36.267935 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:36.267867    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"
Jul 22 19:16:36.401301 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:36.401275    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"]
Jul 22 19:16:36.402968 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:16:36.402932    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c37f7df_2204_4474_a99d_5afb968b5279.slice/crio-14b597632361fa548e04a4a6f0a6c1870ec4348093df6900d003b87664627efa WatchSource:0}: Error finding container 14b597632361fa548e04a4a6f0a6c1870ec4348093df6900d003b87664627efa: Status 404 returned error can't find the container with id 14b597632361fa548e04a4a6f0a6c1870ec4348093df6900d003b87664627efa
Jul 22 19:16:37.376370 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:37.376335    2576 generic.go:356] "Generic (PLEG): container finished" podID="9c37f7df-2204-4474-a99d-5afb968b5279" containerID="3a6a2a9d1461d53ab1ca5ca2b36117567908b7568600e0d2e7818d3639e694b8" exitCode=0
Jul 22 19:16:37.376772 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:37.376397    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv" event={"ID":"9c37f7df-2204-4474-a99d-5afb968b5279","Type":"ContainerDied","Data":"3a6a2a9d1461d53ab1ca5ca2b36117567908b7568600e0d2e7818d3639e694b8"}
Jul 22 19:16:37.376772 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:37.376484    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv" event={"ID":"9c37f7df-2204-4474-a99d-5afb968b5279","Type":"ContainerStarted","Data":"14b597632361fa548e04a4a6f0a6c1870ec4348093df6900d003b87664627efa"}
Jul 22 19:16:38.510518 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:38.510498    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"
Jul 22 19:16:38.591333 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:38.591306    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c37f7df-2204-4474-a99d-5afb968b5279-maas-api-service-ca\") pod \"9c37f7df-2204-4474-a99d-5afb968b5279\" (UID: \"9c37f7df-2204-4474-a99d-5afb968b5279\") "
Jul 22 19:16:38.591659 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:38.591613    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c37f7df-2204-4474-a99d-5afb968b5279-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9c37f7df-2204-4474-a99d-5afb968b5279" (UID: "9c37f7df-2204-4474-a99d-5afb968b5279"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:16:38.692310 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:38.692240    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c37f7df-2204-4474-a99d-5afb968b5279-maas-api-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:16:39.387224 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:39.387193    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"
Jul 22 19:16:39.387386 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:39.387225    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv" event={"ID":"9c37f7df-2204-4474-a99d-5afb968b5279","Type":"ContainerDied","Data":"14b597632361fa548e04a4a6f0a6c1870ec4348093df6900d003b87664627efa"}
Jul 22 19:16:39.387386 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:39.387275    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14b597632361fa548e04a4a6f0a6c1870ec4348093df6900d003b87664627efa"
Jul 22 19:16:46.361043 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.361006    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"]
Jul 22 19:16:46.361499 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.361269    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd" podUID="70960298-5cd8-46a5-a6ce-1ff91e5dc969" containerName="maas-api" containerID="cri-o://d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df" gracePeriod=30
Jul 22 19:16:46.611914 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.611892    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:46.758911 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.758829    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70960298-5cd8-46a5-a6ce-1ff91e5dc969-openshift-service-ca\") pod \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") "
Jul 22 19:16:46.758911 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.758869    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70960298-5cd8-46a5-a6ce-1ff91e5dc969-maas-api-tls\") pod \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") "
Jul 22 19:16:46.759125 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.758960    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mtf6\" (UniqueName: \"kubernetes.io/projected/70960298-5cd8-46a5-a6ce-1ff91e5dc969-kube-api-access-8mtf6\") pod \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\" (UID: \"70960298-5cd8-46a5-a6ce-1ff91e5dc969\") "
Jul 22 19:16:46.759189 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.759161    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70960298-5cd8-46a5-a6ce-1ff91e5dc969-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70960298-5cd8-46a5-a6ce-1ff91e5dc969" (UID: "70960298-5cd8-46a5-a6ce-1ff91e5dc969"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:16:46.760983 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.760961    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70960298-5cd8-46a5-a6ce-1ff91e5dc969-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70960298-5cd8-46a5-a6ce-1ff91e5dc969" (UID: "70960298-5cd8-46a5-a6ce-1ff91e5dc969"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:16:46.761063 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.761043    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70960298-5cd8-46a5-a6ce-1ff91e5dc969-kube-api-access-8mtf6" (OuterVolumeSpecName: "kube-api-access-8mtf6") pod "70960298-5cd8-46a5-a6ce-1ff91e5dc969" (UID: "70960298-5cd8-46a5-a6ce-1ff91e5dc969"). InnerVolumeSpecName "kube-api-access-8mtf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:16:46.859776 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.859729    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mtf6\" (UniqueName: \"kubernetes.io/projected/70960298-5cd8-46a5-a6ce-1ff91e5dc969-kube-api-access-8mtf6\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:16:46.859776 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.859770    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70960298-5cd8-46a5-a6ce-1ff91e5dc969-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:16:46.859776 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:46.859786    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70960298-5cd8-46a5-a6ce-1ff91e5dc969-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:16:47.426204 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.426173    2576 generic.go:356] "Generic (PLEG): container finished" podID="70960298-5cd8-46a5-a6ce-1ff91e5dc969" containerID="d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df" exitCode=0
Jul 22 19:16:47.426596 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.426222    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"
Jul 22 19:16:47.426596 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.426257    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd" event={"ID":"70960298-5cd8-46a5-a6ce-1ff91e5dc969","Type":"ContainerDied","Data":"d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df"}
Jul 22 19:16:47.426596 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.426326    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd" event={"ID":"70960298-5cd8-46a5-a6ce-1ff91e5dc969","Type":"ContainerDied","Data":"208f5afa00c2c6464a665edd17e1eeaeedf22e0d2f9cbb9ccf4e4b7a2f181667"}
Jul 22 19:16:47.426596 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.426361    2576 scope.go:117] "RemoveContainer" containerID="d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df"
Jul 22 19:16:47.436158 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.436129    2576 scope.go:117] "RemoveContainer" containerID="d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df"
Jul 22 19:16:47.436462 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:16:47.436436    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df\": container with ID starting with d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df not found: ID does not exist" containerID="d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df"
Jul 22 19:16:47.436554 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.436470    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df"} err="failed to get container status \"d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df\": rpc error: code = NotFound desc = could not find container \"d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df\": container with ID starting with d33a076a40793657a25a2c3aead731449446988ddeefbd6ecb3cd4ccd1b996df not found: ID does not exist"
Jul 22 19:16:47.449772 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.449748    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"]
Jul 22 19:16:47.453069 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.453049    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3579e215-5bf8bd58f4-pwvhd"]
Jul 22 19:16:47.692750 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:47.692656    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70960298-5cd8-46a5-a6ce-1ff91e5dc969" path="/var/lib/kubelet/pods/70960298-5cd8-46a5-a6ce-1ff91e5dc969/volumes"
Jul 22 19:16:51.031107 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:51.031071    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"]
Jul 22 19:16:51.034129 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:51.034101    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3579e215-7tkhv"]
Jul 22 19:16:51.689667 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:16:51.689612    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c37f7df-2204-4474-a99d-5afb968b5279" path="/var/lib/kubelet/pods/9c37f7df-2204-4474-a99d-5afb968b5279/volumes"
Jul 22 19:17:20.598817 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.598785    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"]
Jul 22 19:17:20.599406 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.599138    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70960298-5cd8-46a5-a6ce-1ff91e5dc969" containerName="maas-api"
Jul 22 19:17:20.599406 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.599149    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="70960298-5cd8-46a5-a6ce-1ff91e5dc969" containerName="maas-api"
Jul 22 19:17:20.599406 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.599157    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c37f7df-2204-4474-a99d-5afb968b5279" containerName="revoke-keys"
Jul 22 19:17:20.599406 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.599164    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c37f7df-2204-4474-a99d-5afb968b5279" containerName="revoke-keys"
Jul 22 19:17:20.599406 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.599229    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="70960298-5cd8-46a5-a6ce-1ff91e5dc969" containerName="maas-api"
Jul 22 19:17:20.599406 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.599235    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c37f7df-2204-4474-a99d-5afb968b5279" containerName="revoke-keys"
Jul 22 19:17:20.602211 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.602191    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:20.606863 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.606844    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-73899c00-serving-cert\""
Jul 22 19:17:20.617740 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.617721    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"]
Jul 22 19:17:20.646365 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.646330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbp78\" (UniqueName: \"kubernetes.io/projected/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-kube-api-access-bbp78\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:20.646483 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.646463    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-openshift-service-ca\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:20.646541 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.646513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-maas-api-tls\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:20.747571 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.747537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-openshift-service-ca\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:20.747939 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.747913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-maas-api-tls\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:20.748011 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.747994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbp78\" (UniqueName: \"kubernetes.io/projected/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-kube-api-access-bbp78\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:20.748094 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:17:20.748080    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-serving-cert: secret "maas-api-e2e-mt-73899c00-serving-cert" not found
Jul 22 19:17:20.748162 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:17:20.748151    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-maas-api-tls podName:ac6185f7-d881-4f07-bbc0-e10bfef6eb9a nodeName:}" failed. No retries permitted until 2026-07-22 19:17:21.248132479 +0000 UTC m=+2242.138673391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-maas-api-tls") pod "maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b" (UID: "ac6185f7-d881-4f07-bbc0-e10bfef6eb9a") : secret "maas-api-e2e-mt-73899c00-serving-cert" not found
Jul 22 19:17:20.748210 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.748159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-openshift-service-ca\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:20.756176 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:20.756151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbp78\" (UniqueName: \"kubernetes.io/projected/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-kube-api-access-bbp78\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:21.253467 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:21.253431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-maas-api-tls\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:21.255943 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:21.255914    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-maas-api-tls\") pod \"maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:21.512941 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:21.512858    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:21.650983 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:21.650959    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"]
Jul 22 19:17:21.654180 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:17:21.654151    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6185f7_d881_4f07_bbc0_e10bfef6eb9a.slice/crio-6e8d745a02b59b923021c2fe68e3a76d983d092ffd6047e2f225b5bfbcd1c700 WatchSource:0}: Error finding container 6e8d745a02b59b923021c2fe68e3a76d983d092ffd6047e2f225b5bfbcd1c700: Status 404 returned error can't find the container with id 6e8d745a02b59b923021c2fe68e3a76d983d092ffd6047e2f225b5bfbcd1c700
Jul 22 19:17:21.655661 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:21.655641    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 19:17:22.610026 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:22.609946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b" event={"ID":"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a","Type":"ContainerStarted","Data":"6e8d745a02b59b923021c2fe68e3a76d983d092ffd6047e2f225b5bfbcd1c700"}
Jul 22 19:17:23.616891 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:23.616853    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b" event={"ID":"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a","Type":"ContainerStarted","Data":"3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c"}
Jul 22 19:17:23.617293 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:23.616928    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:23.633942 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:23.633894    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b" podStartSLOduration=2.525496296 podStartE2EDuration="3.633876267s" totalImagesPullingTime="1.108379971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:17:23.633011082 +0000 UTC m=+2244.523552015" watchObservedRunningTime="2026-07-22 19:17:23.633876267 +0000 UTC m=+2244.524417201"
Jul 22 19:17:29.634729 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:29.634703    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:17:30.791256 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.791225    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"]
Jul 22 19:17:30.795152 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.795132    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:30.797891 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.797870    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-73899c00\"/\"kube-root-ca.crt\""
Jul 22 19:17:30.799062 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.799043    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-73899c00\"/\"e2e-lifecycle-model-73899c00-kserve-self-signed-certs\""
Jul 22 19:17:30.799149 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.799048    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-73899c00\"/\"openshift-service-ca.crt\""
Jul 22 19:17:30.799213 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.799157    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-73899c00\"/\"default-dockercfg-j56zv\""
Jul 22 19:17:30.803606 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.803584    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"]
Jul 22 19:17:30.941497 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.941459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tls-certs\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:30.941696 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.941503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qmv7\" (UniqueName: \"kubernetes.io/projected/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-kube-api-access-4qmv7\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:30.941696 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.941529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-home\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:30.941696 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.941641    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-dshm\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:30.941696 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.941696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tmp-dir\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:30.941891 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:30.941759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-model-cache\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.042700 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.042584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tmp-dir\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.042700 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.042659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-model-cache\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.042700 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.042693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tls-certs\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.042956 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.042722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qmv7\" (UniqueName: \"kubernetes.io/projected/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-kube-api-access-4qmv7\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.042956 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.042749    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-home\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.042956 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.042780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-dshm\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.043088 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.043064    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tmp-dir\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.043134 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.043077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-model-cache\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.043175 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.043164    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-home\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.045036 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.045004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-dshm\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.045246 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.045231    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tls-certs\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.050891 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.050873    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qmv7\" (UniqueName: \"kubernetes.io/projected/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-kube-api-access-4qmv7\") pod \"e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") " pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.105967 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.105928    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:31.243740 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.243712    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"]
Jul 22 19:17:31.246114 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:17:31.246089    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa61c22b_6d8f_4031_b3d0_5ceeebc4f2b3.slice/crio-38841bd268dd44a4a477adbc9c98a1f27a8e6acdf9c8991b59751846297f2c8c WatchSource:0}: Error finding container 38841bd268dd44a4a477adbc9c98a1f27a8e6acdf9c8991b59751846297f2c8c: Status 404 returned error can't find the container with id 38841bd268dd44a4a477adbc9c98a1f27a8e6acdf9c8991b59751846297f2c8c
Jul 22 19:17:31.669293 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.669259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn" event={"ID":"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3","Type":"ContainerStarted","Data":"0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2"}
Jul 22 19:17:31.669293 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.669296    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn" event={"ID":"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3","Type":"ContainerStarted","Data":"38841bd268dd44a4a477adbc9c98a1f27a8e6acdf9c8991b59751846297f2c8c"}
Jul 22 19:17:31.687463 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:31.687412    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn" podStartSLOduration=1.687392558 podStartE2EDuration="1.687392558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:17:31.685705663 +0000 UTC m=+2252.576246596" watchObservedRunningTime="2026-07-22 19:17:31.687392558 +0000 UTC m=+2252.577933490"
Jul 22 19:17:41.106127 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:41.106090    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:41.106609 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:41.106146    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:41.118351 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:41.118328    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:17:41.730838 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:17:41.730809    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:18:29.020983 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.020949    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"]
Jul 22 19:18:29.021836 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.021156    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b" podUID="ac6185f7-d881-4f07-bbc0-e10bfef6eb9a" containerName="maas-api" containerID="cri-o://3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c" gracePeriod=30
Jul 22 19:18:29.265399 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.265375    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:18:29.440476 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.440445    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbp78\" (UniqueName: \"kubernetes.io/projected/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-kube-api-access-bbp78\") pod \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") "
Jul 22 19:18:29.440667 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.440516    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-maas-api-tls\") pod \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") "
Jul 22 19:18:29.440667 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.440547    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-openshift-service-ca\") pod \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\" (UID: \"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a\") "
Jul 22 19:18:29.440967 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.440937    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac6185f7-d881-4f07-bbc0-e10bfef6eb9a" (UID: "ac6185f7-d881-4f07-bbc0-e10bfef6eb9a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:18:29.442703 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.442671    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac6185f7-d881-4f07-bbc0-e10bfef6eb9a" (UID: "ac6185f7-d881-4f07-bbc0-e10bfef6eb9a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:18:29.442703 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.442676    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-kube-api-access-bbp78" (OuterVolumeSpecName: "kube-api-access-bbp78") pod "ac6185f7-d881-4f07-bbc0-e10bfef6eb9a" (UID: "ac6185f7-d881-4f07-bbc0-e10bfef6eb9a"). InnerVolumeSpecName "kube-api-access-bbp78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:18:29.541589 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.541557    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:18:29.541589 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.541584    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbp78\" (UniqueName: \"kubernetes.io/projected/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-kube-api-access-bbp78\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:18:29.541589 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.541594    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:18:29.963949 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.963869    2576 generic.go:356] "Generic (PLEG): container finished" podID="ac6185f7-d881-4f07-bbc0-e10bfef6eb9a" containerID="3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c" exitCode=0
Jul 22 19:18:29.963949 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.963918    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"
Jul 22 19:18:29.964194 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.963947    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b" event={"ID":"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a","Type":"ContainerDied","Data":"3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c"}
Jul 22 19:18:29.964194 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.963997    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b" event={"ID":"ac6185f7-d881-4f07-bbc0-e10bfef6eb9a","Type":"ContainerDied","Data":"6e8d745a02b59b923021c2fe68e3a76d983d092ffd6047e2f225b5bfbcd1c700"}
Jul 22 19:18:29.964194 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.964021    2576 scope.go:117] "RemoveContainer" containerID="3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c"
Jul 22 19:18:29.972527 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.972502    2576 scope.go:117] "RemoveContainer" containerID="3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c"
Jul 22 19:18:29.972804 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:18:29.972785    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c\": container with ID starting with 3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c not found: ID does not exist" containerID="3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c"
Jul 22 19:18:29.972883 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.972815    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c"} err="failed to get container status \"3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c\": rpc error: code = NotFound desc = could not find container \"3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c\": container with ID starting with 3cfc52a9a7bb4302766a283b45039a56504baa5d746758f85cb42cc4e7754d9c not found: ID does not exist"
Jul 22 19:18:29.981439 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.981421    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"]
Jul 22 19:18:29.984290 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:29.984271    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-73899c00-56bb8c4df8-vjn6b"]
Jul 22 19:18:31.687930 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:31.687898    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac6185f7-d881-4f07-bbc0-e10bfef6eb9a" path="/var/lib/kubelet/pods/ac6185f7-d881-4f07-bbc0-e10bfef6eb9a/volumes"
Jul 22 19:18:41.022001 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:41.021968    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"]
Jul 22 19:18:56.043473 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.043383    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn" podUID="fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" containerName="main" containerID="cri-o://0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2" gracePeriod=45
Jul 22 19:18:56.304258 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.304194    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:18:56.364104 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.364073    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-home\") pod \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") "
Jul 22 19:18:56.364295 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.364112    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tmp-dir\") pod \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") "
Jul 22 19:18:56.364295 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.364166    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-model-cache\") pod \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") "
Jul 22 19:18:56.364295 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.364183    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-dshm\") pod \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") "
Jul 22 19:18:56.364295 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.364223    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tls-certs\") pod \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") "
Jul 22 19:18:56.364295 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.364247    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qmv7\" (UniqueName: \"kubernetes.io/projected/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-kube-api-access-4qmv7\") pod \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\" (UID: \"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3\") "
Jul 22 19:18:56.364565 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.364457    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-model-cache" (OuterVolumeSpecName: "model-cache") pod "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" (UID: "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:18:56.364565 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.364401    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-home" (OuterVolumeSpecName: "home") pod "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" (UID: "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:18:56.364696 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.364599    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" (UID: "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:18:56.366667 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.366621    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" (UID: "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:18:56.366852 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.366825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-kube-api-access-4qmv7" (OuterVolumeSpecName: "kube-api-access-4qmv7") pod "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" (UID: "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3"). InnerVolumeSpecName "kube-api-access-4qmv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:18:56.366937 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.366879    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-dshm" (OuterVolumeSpecName: "dshm") pod "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" (UID: "fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:18:56.465932 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.465895    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-model-cache\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:18:56.465932 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.465925    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-dshm\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:18:56.465932 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.465934    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tls-certs\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:18:56.465932 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.465943    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qmv7\" (UniqueName: \"kubernetes.io/projected/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-kube-api-access-4qmv7\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:18:56.466205 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.465951    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-home\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:18:56.466205 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:56.465990    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3-tmp-dir\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:18:57.107890 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.107858    2576 generic.go:356] "Generic (PLEG): container finished" podID="fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" containerID="0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2" exitCode=0
Jul 22 19:18:57.108374 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.107904    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"
Jul 22 19:18:57.108374 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.107951    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn" event={"ID":"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3","Type":"ContainerDied","Data":"0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2"}
Jul 22 19:18:57.108374 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.108002    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn" event={"ID":"fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3","Type":"ContainerDied","Data":"38841bd268dd44a4a477adbc9c98a1f27a8e6acdf9c8991b59751846297f2c8c"}
Jul 22 19:18:57.108374 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.108025    2576 scope.go:117] "RemoveContainer" containerID="0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2"
Jul 22 19:18:57.117075 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.117059    2576 scope.go:117] "RemoveContainer" containerID="0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2"
Jul 22 19:18:57.117347 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:18:57.117327    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2\": container with ID starting with 0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2 not found: ID does not exist" containerID="0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2"
Jul 22 19:18:57.117387 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.117357    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2"} err="failed to get container status \"0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2\": rpc error: code = NotFound desc = could not find container \"0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2\": container with ID starting with 0bf3b415ab74e7c61073385d214485b247c46addad4d44fd0a2d7729bd06b1d2 not found: ID does not exist"
Jul 22 19:18:57.128889 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.128869    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"]
Jul 22 19:18:57.132916 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.132897    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-73899c00/e2e-lifecycle-model-73899c00-kserve-88577595f-w89pn"]
Jul 22 19:18:57.694321 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:57.694287    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" path="/var/lib/kubelet/pods/fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3/volumes"
Jul 22 19:18:59.764832 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:18:59.764735    2576 scope.go:117] "RemoveContainer" containerID="6c20a8f9af903c0cd69c89fc17ea7b13206aca45a9e67805812ac0e6e073fa8a"
Jul 22 19:19:59.790156 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:19:59.790122    2576 scope.go:117] "RemoveContainer" containerID="f76c4c86f4354b2505048502765e98682021ddce388abdd8efad2c4f95dfdc17"
Jul 22 19:20:22.510869 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.510787    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"]
Jul 22 19:20:22.511422 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.511328    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" containerName="main"
Jul 22 19:20:22.511422 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.511347    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" containerName="main"
Jul 22 19:20:22.511422 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.511369    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac6185f7-d881-4f07-bbc0-e10bfef6eb9a" containerName="maas-api"
Jul 22 19:20:22.511422 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.511378    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac6185f7-d881-4f07-bbc0-e10bfef6eb9a" containerName="maas-api"
Jul 22 19:20:22.511655 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.511494    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa61c22b-6d8f-4031-b3d0-5ceeebc4f2b3" containerName="main"
Jul 22 19:20:22.511655 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.511512    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac6185f7-d881-4f07-bbc0-e10bfef6eb9a" containerName="maas-api"
Jul 22 19:20:22.514528 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.514509    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:22.517572 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.517549    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c4b608-serving-cert\""
Jul 22 19:20:22.524452 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.524427    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"]
Jul 22 19:20:22.596726 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.596694    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4drs\" (UniqueName: \"kubernetes.io/projected/07762366-b844-4d4a-a389-27844dc25fb5-kube-api-access-r4drs\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:22.596863 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.596736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:22.596863 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.596756    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07762366-b844-4d4a-a389-27844dc25fb5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:22.698153 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.698125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4drs\" (UniqueName: \"kubernetes.io/projected/07762366-b844-4d4a-a389-27844dc25fb5-kube-api-access-r4drs\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:22.698153 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.698166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:22.698351 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.698192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07762366-b844-4d4a-a389-27844dc25fb5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:22.698351 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:20:22.698276    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-serving-cert: secret "maas-api-e2e-inf-a-c4b608-serving-cert" not found
Jul 22 19:20:22.698351 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:20:22.698331    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls podName:07762366-b844-4d4a-a389-27844dc25fb5 nodeName:}" failed. No retries permitted until 2026-07-22 19:20:23.198316619 +0000 UTC m=+2424.088857528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls") pod "maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7" (UID: "07762366-b844-4d4a-a389-27844dc25fb5") : secret "maas-api-e2e-inf-a-c4b608-serving-cert" not found
Jul 22 19:20:22.698843 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.698825    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07762366-b844-4d4a-a389-27844dc25fb5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:22.707802 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:22.707781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4drs\" (UniqueName: \"kubernetes.io/projected/07762366-b844-4d4a-a389-27844dc25fb5-kube-api-access-r4drs\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:23.202383 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:23.202343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:23.202557 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:20:23.202487    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-serving-cert: secret "maas-api-e2e-inf-a-c4b608-serving-cert" not found
Jul 22 19:20:23.202557 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:20:23.202550    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls podName:07762366-b844-4d4a-a389-27844dc25fb5 nodeName:}" failed. No retries permitted until 2026-07-22 19:20:24.202532765 +0000 UTC m=+2425.093073678 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls") pod "maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7" (UID: "07762366-b844-4d4a-a389-27844dc25fb5") : secret "maas-api-e2e-inf-a-c4b608-serving-cert" not found
Jul 22 19:20:24.211774 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:24.211736    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:24.214209 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:24.214186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:24.326175 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:24.326136    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:24.459574 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:24.459544    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"]
Jul 22 19:20:24.461544 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:20:24.461518    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07762366_b844_4d4a_a389_27844dc25fb5.slice/crio-97f4f454d1eb90d51edc35e88b61d127da93e11adb28d1bcf0f2e9a96e3bcf10 WatchSource:0}: Error finding container 97f4f454d1eb90d51edc35e88b61d127da93e11adb28d1bcf0f2e9a96e3bcf10: Status 404 returned error can't find the container with id 97f4f454d1eb90d51edc35e88b61d127da93e11adb28d1bcf0f2e9a96e3bcf10
Jul 22 19:20:24.531312 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:24.531267    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7" event={"ID":"07762366-b844-4d4a-a389-27844dc25fb5","Type":"ContainerStarted","Data":"97f4f454d1eb90d51edc35e88b61d127da93e11adb28d1bcf0f2e9a96e3bcf10"}
Jul 22 19:20:25.538697 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:25.538659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7" event={"ID":"07762366-b844-4d4a-a389-27844dc25fb5","Type":"ContainerStarted","Data":"6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5"}
Jul 22 19:20:25.539102 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:25.538723    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:25.558379 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:25.558324    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7" podStartSLOduration=3.103150644 podStartE2EDuration="3.558307995s" totalImagesPullingTime="455.157351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:20:25.556334042 +0000 UTC m=+2426.446874975" watchObservedRunningTime="2026-07-22 19:20:25.558307995 +0000 UTC m=+2426.448848926"
Jul 22 19:20:30.536362 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.536323    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"]
Jul 22 19:20:30.540903 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.540880    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.543701 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.543682    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ba9716-serving-cert\""
Jul 22 19:20:30.551961 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.551937    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"]
Jul 22 19:20:30.670824 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.670791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh5ns\" (UniqueName: \"kubernetes.io/projected/151eefd4-8f8d-422c-862a-68816be8bf84-kube-api-access-vh5ns\") pod \"maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.671007 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.670845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/151eefd4-8f8d-422c-862a-68816be8bf84-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.671007 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.670938    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/151eefd4-8f8d-422c-862a-68816be8bf84-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.772246 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.772217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/151eefd4-8f8d-422c-862a-68816be8bf84-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.772463 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.772278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh5ns\" (UniqueName: \"kubernetes.io/projected/151eefd4-8f8d-422c-862a-68816be8bf84-kube-api-access-vh5ns\") pod \"maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.772463 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.772312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/151eefd4-8f8d-422c-862a-68816be8bf84-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.772898 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.772871    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/151eefd4-8f8d-422c-862a-68816be8bf84-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.774743 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.774725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/151eefd4-8f8d-422c-862a-68816be8bf84-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.782893 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.782871    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh5ns\" (UniqueName: \"kubernetes.io/projected/151eefd4-8f8d-422c-862a-68816be8bf84-kube-api-access-vh5ns\") pod \"maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.852724 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.852653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:30.992455 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:30.992431    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"]
Jul 22 19:20:30.994561 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:20:30.994535    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151eefd4_8f8d_422c_862a_68816be8bf84.slice/crio-1ca23310ec8a66950d44bad01e98f3cbc46d39bb5296c6608e1f818d009342ad WatchSource:0}: Error finding container 1ca23310ec8a66950d44bad01e98f3cbc46d39bb5296c6608e1f818d009342ad: Status 404 returned error can't find the container with id 1ca23310ec8a66950d44bad01e98f3cbc46d39bb5296c6608e1f818d009342ad
Jul 22 19:20:31.550261 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:31.550235    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:20:31.571963 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:31.571927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c" event={"ID":"151eefd4-8f8d-422c-862a-68816be8bf84","Type":"ContainerStarted","Data":"c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651"}
Jul 22 19:20:31.572307 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:31.571976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c" event={"ID":"151eefd4-8f8d-422c-862a-68816be8bf84","Type":"ContainerStarted","Data":"1ca23310ec8a66950d44bad01e98f3cbc46d39bb5296c6608e1f818d009342ad"}
Jul 22 19:20:31.572307 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:31.572089    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:31.590481 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:31.590432    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c" podStartSLOduration=1.175027054 podStartE2EDuration="1.590418542s" totalImagesPullingTime="415.391488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:20:31.589119513 +0000 UTC m=+2432.479660447" watchObservedRunningTime="2026-07-22 19:20:31.590418542 +0000 UTC m=+2432.480959474"
Jul 22 19:20:37.582903 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:37.582873    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:20:42.692605 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.692574    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"]
Jul 22 19:20:42.696051 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.696034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.699968 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.699936    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c4b608\"/\"openshift-service-ca.crt\""
Jul 22 19:20:42.700086 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.700019    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c4b608\"/\"test-model-c4b608-kserve-self-signed-certs\""
Jul 22 19:20:42.700086 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.700023    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c4b608\"/\"default-dockercfg-zgppw\""
Jul 22 19:20:42.700171 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.700023    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c4b608\"/\"kube-root-ca.crt\""
Jul 22 19:20:42.706154 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.706131    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"]
Jul 22 19:20:42.782600 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.782569    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-home\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.782773 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.782659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk7sw\" (UniqueName: \"kubernetes.io/projected/80ce0a90-1d33-4588-8a08-edaa64a8a090-kube-api-access-gk7sw\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.782773 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.782710    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-dshm\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.782773 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.782726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ce0a90-1d33-4588-8a08-edaa64a8a090-tls-certs\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.782909 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.782791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-model-cache\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.782909 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.782824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-tmp-dir\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.883818 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.883782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-home\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.883990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.883850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk7sw\" (UniqueName: \"kubernetes.io/projected/80ce0a90-1d33-4588-8a08-edaa64a8a090-kube-api-access-gk7sw\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.883990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.883894    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-dshm\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.883990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.883911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ce0a90-1d33-4588-8a08-edaa64a8a090-tls-certs\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.883990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.883938    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-model-cache\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.883990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.883970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-tmp-dir\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.884238 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.884176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-home\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.884317 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.884299    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-tmp-dir\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.884428 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.884408    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-model-cache\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.886210 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.886189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-dshm\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.886356 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.886341    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ce0a90-1d33-4588-8a08-edaa64a8a090-tls-certs\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:42.891216 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:42.891196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk7sw\" (UniqueName: \"kubernetes.io/projected/80ce0a90-1d33-4588-8a08-edaa64a8a090-kube-api-access-gk7sw\") pod \"test-model-c4b608-kserve-6866685ddc-sbtpk\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") " pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:43.006387 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:43.006302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:43.144465 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:43.144431    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"]
Jul 22 19:20:43.146606 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:20:43.146575    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80ce0a90_1d33_4588_8a08_edaa64a8a090.slice/crio-aff5bd87564df610488d4c27012a845afe5f4729502c73fdcb13114c2392ef30 WatchSource:0}: Error finding container aff5bd87564df610488d4c27012a845afe5f4729502c73fdcb13114c2392ef30: Status 404 returned error can't find the container with id aff5bd87564df610488d4c27012a845afe5f4729502c73fdcb13114c2392ef30
Jul 22 19:20:43.633317 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:43.633283    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk" event={"ID":"80ce0a90-1d33-4588-8a08-edaa64a8a090","Type":"ContainerStarted","Data":"9d6426215f5be1b2a6c9dbcd3672397b668136a16e386962aea23d757c3f62ea"}
Jul 22 19:20:43.633526 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:43.633328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk" event={"ID":"80ce0a90-1d33-4588-8a08-edaa64a8a090","Type":"ContainerStarted","Data":"aff5bd87564df610488d4c27012a845afe5f4729502c73fdcb13114c2392ef30"}
Jul 22 19:20:43.649483 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:43.649435    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk" podStartSLOduration=1.6494206340000002 podStartE2EDuration="1.649420634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:20:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:20:43.647884584 +0000 UTC m=+2444.538425555" watchObservedRunningTime="2026-07-22 19:20:43.649420634 +0000 UTC m=+2444.539961565"
Jul 22 19:20:53.007219 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:53.007179    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:53.007943 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:53.007253    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:53.019967 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:53.019943    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:53.701552 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:53.701522    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:20:59.808522 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:20:59.808484    2576 scope.go:117] "RemoveContainer" containerID="ceeb106b07be2739cc352b04b06920358810956c77b9115ce4402fa33db68b19"
Jul 22 19:22:04.907159 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:04.907069    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"]
Jul 22 19:22:06.061239 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:06.061198    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"]
Jul 22 19:22:06.065383 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:06.065363    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"
Jul 22 19:22:06.068524 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:06.068499    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5ttk\""
Jul 22 19:22:06.071004 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:06.070980    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"]
Jul 22 19:22:06.198854 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:06.198824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab3b4cb-5752-4e8f-8239-34f6fc269906-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws\" (UID: \"aab3b4cb-5752-4e8f-8239-34f6fc269906\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"
Jul 22 19:22:06.299432 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:06.299396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab3b4cb-5752-4e8f-8239-34f6fc269906-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws\" (UID: \"aab3b4cb-5752-4e8f-8239-34f6fc269906\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"
Jul 22 19:22:06.300003 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:06.299983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab3b4cb-5752-4e8f-8239-34f6fc269906-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws\" (UID: \"aab3b4cb-5752-4e8f-8239-34f6fc269906\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"
Jul 22 19:22:06.379874 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:06.379849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"
Jul 22 19:22:06.515721 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:06.515696    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"]
Jul 22 19:22:06.518431 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:22:06.518401    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab3b4cb_5752_4e8f_8239_34f6fc269906.slice/crio-0dc2ff568f7b3ee8e7771dbfedffe4aae944cd35b6343090f5b2dada3d608359 WatchSource:0}: Error finding container 0dc2ff568f7b3ee8e7771dbfedffe4aae944cd35b6343090f5b2dada3d608359: Status 404 returned error can't find the container with id 0dc2ff568f7b3ee8e7771dbfedffe4aae944cd35b6343090f5b2dada3d608359
Jul 22 19:22:07.043623 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:07.043588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws" event={"ID":"aab3b4cb-5752-4e8f-8239-34f6fc269906","Type":"ContainerStarted","Data":"5e3a9d1623dc9ac85850c6b22cc0495433e9c2e0d982ecc986fa6cb5595955d3"}
Jul 22 19:22:07.043808 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:07.043681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws" event={"ID":"aab3b4cb-5752-4e8f-8239-34f6fc269906","Type":"ContainerStarted","Data":"0dc2ff568f7b3ee8e7771dbfedffe4aae944cd35b6343090f5b2dada3d608359"}
Jul 22 19:22:07.066346 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:07.066290    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws" podStartSLOduration=1.06627384 podStartE2EDuration="1.06627384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:22:07.05787268 +0000 UTC m=+2527.948413613" watchObservedRunningTime="2026-07-22 19:22:07.06627384 +0000 UTC m=+2527.956814772"
Jul 22 19:22:08.050222 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:08.050184    2576 generic.go:356] "Generic (PLEG): container finished" podID="aab3b4cb-5752-4e8f-8239-34f6fc269906" containerID="5e3a9d1623dc9ac85850c6b22cc0495433e9c2e0d982ecc986fa6cb5595955d3" exitCode=0
Jul 22 19:22:08.050374 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:08.050251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws" event={"ID":"aab3b4cb-5752-4e8f-8239-34f6fc269906","Type":"ContainerDied","Data":"5e3a9d1623dc9ac85850c6b22cc0495433e9c2e0d982ecc986fa6cb5595955d3"}
Jul 22 19:22:09.186916 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:09.186892    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"
Jul 22 19:22:09.325511 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:09.325429    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab3b4cb-5752-4e8f-8239-34f6fc269906-maas-api-service-ca\") pod \"aab3b4cb-5752-4e8f-8239-34f6fc269906\" (UID: \"aab3b4cb-5752-4e8f-8239-34f6fc269906\") "
Jul 22 19:22:09.325816 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:09.325792    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aab3b4cb-5752-4e8f-8239-34f6fc269906-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aab3b4cb-5752-4e8f-8239-34f6fc269906" (UID: "aab3b4cb-5752-4e8f-8239-34f6fc269906"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:22:09.426456 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:09.426428    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab3b4cb-5752-4e8f-8239-34f6fc269906-maas-api-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:10.061366 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:10.061279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws" event={"ID":"aab3b4cb-5752-4e8f-8239-34f6fc269906","Type":"ContainerDied","Data":"0dc2ff568f7b3ee8e7771dbfedffe4aae944cd35b6343090f5b2dada3d608359"}
Jul 22 19:22:10.061366 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:10.061335    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dc2ff568f7b3ee8e7771dbfedffe4aae944cd35b6343090f5b2dada3d608359"
Jul 22 19:22:10.061552 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:10.061291    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"
Jul 22 19:22:16.541328 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.541285    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"]
Jul 22 19:22:16.541961 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.541572    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7" podUID="07762366-b844-4d4a-a389-27844dc25fb5" containerName="maas-api" containerID="cri-o://6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5" gracePeriod=30
Jul 22 19:22:16.795078 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.795018    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:22:16.891068 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.891031    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07762366-b844-4d4a-a389-27844dc25fb5-openshift-service-ca\") pod \"07762366-b844-4d4a-a389-27844dc25fb5\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") "
Jul 22 19:22:16.891068 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.891072    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls\") pod \"07762366-b844-4d4a-a389-27844dc25fb5\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") "
Jul 22 19:22:16.891270 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.891110    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4drs\" (UniqueName: \"kubernetes.io/projected/07762366-b844-4d4a-a389-27844dc25fb5-kube-api-access-r4drs\") pod \"07762366-b844-4d4a-a389-27844dc25fb5\" (UID: \"07762366-b844-4d4a-a389-27844dc25fb5\") "
Jul 22 19:22:16.891457 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.891435    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07762366-b844-4d4a-a389-27844dc25fb5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "07762366-b844-4d4a-a389-27844dc25fb5" (UID: "07762366-b844-4d4a-a389-27844dc25fb5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:22:16.893294 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.893270    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "07762366-b844-4d4a-a389-27844dc25fb5" (UID: "07762366-b844-4d4a-a389-27844dc25fb5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:22:16.893294 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.893283    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07762366-b844-4d4a-a389-27844dc25fb5-kube-api-access-r4drs" (OuterVolumeSpecName: "kube-api-access-r4drs") pod "07762366-b844-4d4a-a389-27844dc25fb5" (UID: "07762366-b844-4d4a-a389-27844dc25fb5"). InnerVolumeSpecName "kube-api-access-r4drs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:22:16.992072 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.992047    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07762366-b844-4d4a-a389-27844dc25fb5-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:16.992072 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.992070    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07762366-b844-4d4a-a389-27844dc25fb5-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:16.992072 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:16.992079    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4drs\" (UniqueName: \"kubernetes.io/projected/07762366-b844-4d4a-a389-27844dc25fb5-kube-api-access-r4drs\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:17.096964 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.096892    2576 generic.go:356] "Generic (PLEG): container finished" podID="07762366-b844-4d4a-a389-27844dc25fb5" containerID="6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5" exitCode=0
Jul 22 19:22:17.096964 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.096939    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"
Jul 22 19:22:17.096964 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.096955    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7" event={"ID":"07762366-b844-4d4a-a389-27844dc25fb5","Type":"ContainerDied","Data":"6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5"}
Jul 22 19:22:17.097211 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.097014    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7" event={"ID":"07762366-b844-4d4a-a389-27844dc25fb5","Type":"ContainerDied","Data":"97f4f454d1eb90d51edc35e88b61d127da93e11adb28d1bcf0f2e9a96e3bcf10"}
Jul 22 19:22:17.097211 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.097045    2576 scope.go:117] "RemoveContainer" containerID="6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5"
Jul 22 19:22:17.106946 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.106926    2576 scope.go:117] "RemoveContainer" containerID="6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5"
Jul 22 19:22:17.107198 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:22:17.107177    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5\": container with ID starting with 6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5 not found: ID does not exist" containerID="6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5"
Jul 22 19:22:17.107246 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.107206    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5"} err="failed to get container status \"6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5\": rpc error: code = NotFound desc = could not find container \"6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5\": container with ID starting with 6999a01a03c16e931d905226ef699d733a67df632f9f54406073fb3d74bd80d5 not found: ID does not exist"
Jul 22 19:22:17.118349 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.118328    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"]
Jul 22 19:22:17.122609 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.122582    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c4b608-6687fc9c9d-49fl7"]
Jul 22 19:22:17.688006 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:17.687946    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07762366-b844-4d4a-a389-27844dc25fb5" path="/var/lib/kubelet/pods/07762366-b844-4d4a-a389-27844dc25fb5/volumes"
Jul 22 19:22:19.927399 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:19.927357    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk" podUID="80ce0a90-1d33-4588-8a08-edaa64a8a090" containerName="main" containerID="cri-o://9d6426215f5be1b2a6c9dbcd3672397b668136a16e386962aea23d757c3f62ea" gracePeriod=45
Jul 22 19:22:20.114538 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.114510    2576 generic.go:356] "Generic (PLEG): container finished" podID="80ce0a90-1d33-4588-8a08-edaa64a8a090" containerID="9d6426215f5be1b2a6c9dbcd3672397b668136a16e386962aea23d757c3f62ea" exitCode=0
Jul 22 19:22:20.114680 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.114578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk" event={"ID":"80ce0a90-1d33-4588-8a08-edaa64a8a090","Type":"ContainerDied","Data":"9d6426215f5be1b2a6c9dbcd3672397b668136a16e386962aea23d757c3f62ea"}
Jul 22 19:22:20.185413 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.185358    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:22:20.321137 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321107    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk7sw\" (UniqueName: \"kubernetes.io/projected/80ce0a90-1d33-4588-8a08-edaa64a8a090-kube-api-access-gk7sw\") pod \"80ce0a90-1d33-4588-8a08-edaa64a8a090\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") "
Jul 22 19:22:20.321330 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321154    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-home\") pod \"80ce0a90-1d33-4588-8a08-edaa64a8a090\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") "
Jul 22 19:22:20.321330 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321226    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ce0a90-1d33-4588-8a08-edaa64a8a090-tls-certs\") pod \"80ce0a90-1d33-4588-8a08-edaa64a8a090\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") "
Jul 22 19:22:20.321330 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321249    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-tmp-dir\") pod \"80ce0a90-1d33-4588-8a08-edaa64a8a090\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") "
Jul 22 19:22:20.321330 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321273    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-model-cache\") pod \"80ce0a90-1d33-4588-8a08-edaa64a8a090\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") "
Jul 22 19:22:20.321330 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321321    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-dshm\") pod \"80ce0a90-1d33-4588-8a08-edaa64a8a090\" (UID: \"80ce0a90-1d33-4588-8a08-edaa64a8a090\") "
Jul 22 19:22:20.321611 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321517    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-home" (OuterVolumeSpecName: "home") pod "80ce0a90-1d33-4588-8a08-edaa64a8a090" (UID: "80ce0a90-1d33-4588-8a08-edaa64a8a090"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:22:20.321611 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321555    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80ce0a90-1d33-4588-8a08-edaa64a8a090" (UID: "80ce0a90-1d33-4588-8a08-edaa64a8a090"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:22:20.321611 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321592    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-model-cache" (OuterVolumeSpecName: "model-cache") pod "80ce0a90-1d33-4588-8a08-edaa64a8a090" (UID: "80ce0a90-1d33-4588-8a08-edaa64a8a090"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:22:20.321782 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321762    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-home\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:20.321819 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321789    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-tmp-dir\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:20.321819 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.321806    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-model-cache\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:20.323365 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.323335    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-dshm" (OuterVolumeSpecName: "dshm") pod "80ce0a90-1d33-4588-8a08-edaa64a8a090" (UID: "80ce0a90-1d33-4588-8a08-edaa64a8a090"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:22:20.323465 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.323380    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80ce0a90-1d33-4588-8a08-edaa64a8a090-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80ce0a90-1d33-4588-8a08-edaa64a8a090" (UID: "80ce0a90-1d33-4588-8a08-edaa64a8a090"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:22:20.323465 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.323411    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ce0a90-1d33-4588-8a08-edaa64a8a090-kube-api-access-gk7sw" (OuterVolumeSpecName: "kube-api-access-gk7sw") pod "80ce0a90-1d33-4588-8a08-edaa64a8a090" (UID: "80ce0a90-1d33-4588-8a08-edaa64a8a090"). InnerVolumeSpecName "kube-api-access-gk7sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:22:20.423161 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.423132    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk7sw\" (UniqueName: \"kubernetes.io/projected/80ce0a90-1d33-4588-8a08-edaa64a8a090-kube-api-access-gk7sw\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:20.423161 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.423158    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80ce0a90-1d33-4588-8a08-edaa64a8a090-tls-certs\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:20.423304 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:20.423172    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80ce0a90-1d33-4588-8a08-edaa64a8a090-dshm\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:21.121679 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:21.121651    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"
Jul 22 19:22:21.122084 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:21.121685    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk" event={"ID":"80ce0a90-1d33-4588-8a08-edaa64a8a090","Type":"ContainerDied","Data":"aff5bd87564df610488d4c27012a845afe5f4729502c73fdcb13114c2392ef30"}
Jul 22 19:22:21.122084 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:21.121738    2576 scope.go:117] "RemoveContainer" containerID="9d6426215f5be1b2a6c9dbcd3672397b668136a16e386962aea23d757c3f62ea"
Jul 22 19:22:21.143819 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:21.143795    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"]
Jul 22 19:22:21.148844 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:21.148824    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c4b608/test-model-c4b608-kserve-6866685ddc-sbtpk"]
Jul 22 19:22:21.155777 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:21.155757    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"]
Jul 22 19:22:21.160825 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:21.160805    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c4b608-nc8ws"]
Jul 22 19:22:21.688600 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:21.688570    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ce0a90-1d33-4588-8a08-edaa64a8a090" path="/var/lib/kubelet/pods/80ce0a90-1d33-4588-8a08-edaa64a8a090/volumes"
Jul 22 19:22:21.688917 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:21.688903    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aab3b4cb-5752-4e8f-8239-34f6fc269906" path="/var/lib/kubelet/pods/aab3b4cb-5752-4e8f-8239-34f6fc269906/volumes"
Jul 22 19:22:38.408615 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.407970    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"]
Jul 22 19:22:38.408615 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.408227    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c" podUID="151eefd4-8f8d-422c-862a-68816be8bf84" containerName="maas-api" containerID="cri-o://c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651" gracePeriod=30
Jul 22 19:22:38.669853 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.669801    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:22:38.762574 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.762545    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/151eefd4-8f8d-422c-862a-68816be8bf84-maas-api-tls\") pod \"151eefd4-8f8d-422c-862a-68816be8bf84\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") "
Jul 22 19:22:38.762732 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.762583    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh5ns\" (UniqueName: \"kubernetes.io/projected/151eefd4-8f8d-422c-862a-68816be8bf84-kube-api-access-vh5ns\") pod \"151eefd4-8f8d-422c-862a-68816be8bf84\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") "
Jul 22 19:22:38.762732 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.762614    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/151eefd4-8f8d-422c-862a-68816be8bf84-openshift-service-ca\") pod \"151eefd4-8f8d-422c-862a-68816be8bf84\" (UID: \"151eefd4-8f8d-422c-862a-68816be8bf84\") "
Jul 22 19:22:38.763058 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.763033    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/151eefd4-8f8d-422c-862a-68816be8bf84-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "151eefd4-8f8d-422c-862a-68816be8bf84" (UID: "151eefd4-8f8d-422c-862a-68816be8bf84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:22:38.764568 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.764545    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/151eefd4-8f8d-422c-862a-68816be8bf84-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "151eefd4-8f8d-422c-862a-68816be8bf84" (UID: "151eefd4-8f8d-422c-862a-68816be8bf84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:22:38.764790 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.764766    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/151eefd4-8f8d-422c-862a-68816be8bf84-kube-api-access-vh5ns" (OuterVolumeSpecName: "kube-api-access-vh5ns") pod "151eefd4-8f8d-422c-862a-68816be8bf84" (UID: "151eefd4-8f8d-422c-862a-68816be8bf84"). InnerVolumeSpecName "kube-api-access-vh5ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:22:38.863862 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.863834    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/151eefd4-8f8d-422c-862a-68816be8bf84-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:38.864008 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.863908    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/151eefd4-8f8d-422c-862a-68816be8bf84-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:38.864008 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:38.863928    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh5ns\" (UniqueName: \"kubernetes.io/projected/151eefd4-8f8d-422c-862a-68816be8bf84-kube-api-access-vh5ns\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:22:39.212352 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.212316    2576 generic.go:356] "Generic (PLEG): container finished" podID="151eefd4-8f8d-422c-862a-68816be8bf84" containerID="c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651" exitCode=0
Jul 22 19:22:39.212521 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.212366    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"
Jul 22 19:22:39.212521 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.212391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c" event={"ID":"151eefd4-8f8d-422c-862a-68816be8bf84","Type":"ContainerDied","Data":"c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651"}
Jul 22 19:22:39.212521 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.212448    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c" event={"ID":"151eefd4-8f8d-422c-862a-68816be8bf84","Type":"ContainerDied","Data":"1ca23310ec8a66950d44bad01e98f3cbc46d39bb5296c6608e1f818d009342ad"}
Jul 22 19:22:39.212521 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.212470    2576 scope.go:117] "RemoveContainer" containerID="c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651"
Jul 22 19:22:39.221620 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.221605    2576 scope.go:117] "RemoveContainer" containerID="c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651"
Jul 22 19:22:39.221874 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:22:39.221856    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651\": container with ID starting with c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651 not found: ID does not exist" containerID="c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651"
Jul 22 19:22:39.221938 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.221883    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651"} err="failed to get container status \"c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651\": rpc error: code = NotFound desc = could not find container \"c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651\": container with ID starting with c4ba2d3bc069eda6681cba44570e366186c870321c55f28adeb5c02d22f0a651 not found: ID does not exist"
Jul 22 19:22:39.234003 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.233978    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"]
Jul 22 19:22:39.238247 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.238225    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba9716-7d656f675b-ctk2c"]
Jul 22 19:22:39.688634 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:39.688605    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="151eefd4-8f8d-422c-862a-68816be8bf84" path="/var/lib/kubelet/pods/151eefd4-8f8d-422c-862a-68816be8bf84/volumes"
Jul 22 19:22:53.431369 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431336    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"]
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431722    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="151eefd4-8f8d-422c-862a-68816be8bf84" containerName="maas-api"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431737    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="151eefd4-8f8d-422c-862a-68816be8bf84" containerName="maas-api"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431752    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ce0a90-1d33-4588-8a08-edaa64a8a090" containerName="main"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431759    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ce0a90-1d33-4588-8a08-edaa64a8a090" containerName="main"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431787    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07762366-b844-4d4a-a389-27844dc25fb5" containerName="maas-api"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431796    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="07762366-b844-4d4a-a389-27844dc25fb5" containerName="maas-api"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431810    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aab3b4cb-5752-4e8f-8239-34f6fc269906" containerName="revoke-keys"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431819    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aab3b4cb-5752-4e8f-8239-34f6fc269906" containerName="revoke-keys"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431908    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ce0a90-1d33-4588-8a08-edaa64a8a090" containerName="main"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431917    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="07762366-b844-4d4a-a389-27844dc25fb5" containerName="maas-api"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431927    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aab3b4cb-5752-4e8f-8239-34f6fc269906" containerName="revoke-keys"
Jul 22 19:22:53.431918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.431939    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="151eefd4-8f8d-422c-862a-68816be8bf84" containerName="maas-api"
Jul 22 19:22:53.436469 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.436449    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.439286 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.439266    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6c9526-serving-cert\""
Jul 22 19:22:53.447261 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.447240    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"]
Jul 22 19:22:53.487799 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.487775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-maas-api-tls\") pod \"maas-api-e2e-api-a-6c9526-75c887f949-6lrsx\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.487902 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.487814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn8k7\" (UniqueName: \"kubernetes.io/projected/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-kube-api-access-vn8k7\") pod \"maas-api-e2e-api-a-6c9526-75c887f949-6lrsx\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.487902 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.487868    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-openshift-service-ca\") pod \"maas-api-e2e-api-a-6c9526-75c887f949-6lrsx\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.589042 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.589013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-maas-api-tls\") pod \"maas-api-e2e-api-a-6c9526-75c887f949-6lrsx\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.589200 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.589059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn8k7\" (UniqueName: \"kubernetes.io/projected/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-kube-api-access-vn8k7\") pod \"maas-api-e2e-api-a-6c9526-75c887f949-6lrsx\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.589200 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.589099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-openshift-service-ca\") pod \"maas-api-e2e-api-a-6c9526-75c887f949-6lrsx\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.589677 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.589655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-openshift-service-ca\") pod \"maas-api-e2e-api-a-6c9526-75c887f949-6lrsx\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.591434 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.591417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-maas-api-tls\") pod \"maas-api-e2e-api-a-6c9526-75c887f949-6lrsx\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.597078 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.597059    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn8k7\" (UniqueName: \"kubernetes.io/projected/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-kube-api-access-vn8k7\") pod \"maas-api-e2e-api-a-6c9526-75c887f949-6lrsx\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.750270 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.750197    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:53.897555 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.897530    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"]
Jul 22 19:22:53.899866 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:22:53.899836    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca7fb2f_0145_41ac_93f4_9c4a9efb6231.slice/crio-9de7a7f9b7cb3467f4f19532f0f0c25b055276e0b9b9e85c3ed7f0f1a0933e3e WatchSource:0}: Error finding container 9de7a7f9b7cb3467f4f19532f0f0c25b055276e0b9b9e85c3ed7f0f1a0933e3e: Status 404 returned error can't find the container with id 9de7a7f9b7cb3467f4f19532f0f0c25b055276e0b9b9e85c3ed7f0f1a0933e3e
Jul 22 19:22:53.901211 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:53.901194    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 19:22:54.286382 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:54.286340    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx" event={"ID":"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231","Type":"ContainerStarted","Data":"9de7a7f9b7cb3467f4f19532f0f0c25b055276e0b9b9e85c3ed7f0f1a0933e3e"}
Jul 22 19:22:55.292776 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:55.292745    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx" event={"ID":"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231","Type":"ContainerStarted","Data":"874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288"}
Jul 22 19:22:55.293143 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:55.292885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:22:55.310835 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:55.310791    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx" podStartSLOduration=1.719458838 podStartE2EDuration="2.310778121s" totalImagesPullingTime="591.319283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:22:55.309270462 +0000 UTC m=+2576.199811395" watchObservedRunningTime="2026-07-22 19:22:55.310778121 +0000 UTC m=+2576.201319053"
Jul 22 19:22:59.837616 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:22:59.837582    2576 scope.go:117] "RemoveContainer" containerID="3a6a2a9d1461d53ab1ca5ca2b36117567908b7568600e0d2e7818d3639e694b8"
Jul 22 19:23:01.303457 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.303432    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:23:01.413479 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.413448    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"]
Jul 22 19:23:01.417167 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.417143    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.419815 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.419797    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-65c717-serving-cert\""
Jul 22 19:23:01.428300 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.428276    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"]
Jul 22 19:23:01.457481 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.457449    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e799479-db0f-470b-acf3-414803226167-maas-api-tls\") pod \"maas-api-e2e-api-b-65c717-6bc5699788-wx49k\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.457659 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.457487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e799479-db0f-470b-acf3-414803226167-openshift-service-ca\") pod \"maas-api-e2e-api-b-65c717-6bc5699788-wx49k\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.457659 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.457507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvkn4\" (UniqueName: \"kubernetes.io/projected/2e799479-db0f-470b-acf3-414803226167-kube-api-access-qvkn4\") pod \"maas-api-e2e-api-b-65c717-6bc5699788-wx49k\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.559157 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.559077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e799479-db0f-470b-acf3-414803226167-maas-api-tls\") pod \"maas-api-e2e-api-b-65c717-6bc5699788-wx49k\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.559157 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.559116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e799479-db0f-470b-acf3-414803226167-openshift-service-ca\") pod \"maas-api-e2e-api-b-65c717-6bc5699788-wx49k\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.559157 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.559140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvkn4\" (UniqueName: \"kubernetes.io/projected/2e799479-db0f-470b-acf3-414803226167-kube-api-access-qvkn4\") pod \"maas-api-e2e-api-b-65c717-6bc5699788-wx49k\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.559862 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.559818    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e799479-db0f-470b-acf3-414803226167-openshift-service-ca\") pod \"maas-api-e2e-api-b-65c717-6bc5699788-wx49k\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.561666 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.561615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e799479-db0f-470b-acf3-414803226167-maas-api-tls\") pod \"maas-api-e2e-api-b-65c717-6bc5699788-wx49k\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.567523 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.567492    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvkn4\" (UniqueName: \"kubernetes.io/projected/2e799479-db0f-470b-acf3-414803226167-kube-api-access-qvkn4\") pod \"maas-api-e2e-api-b-65c717-6bc5699788-wx49k\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.729362 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.729329    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:01.867355 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:01.867329    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"]
Jul 22 19:23:01.869358 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:23:01.869329    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e799479_db0f_470b_acf3_414803226167.slice/crio-133401a5b454963f96d971a5783669b63c529a75433cdc3544eacd049b4aba6f WatchSource:0}: Error finding container 133401a5b454963f96d971a5783669b63c529a75433cdc3544eacd049b4aba6f: Status 404 returned error can't find the container with id 133401a5b454963f96d971a5783669b63c529a75433cdc3544eacd049b4aba6f
Jul 22 19:23:02.349182 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:02.349140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k" event={"ID":"2e799479-db0f-470b-acf3-414803226167","Type":"ContainerStarted","Data":"1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f"}
Jul 22 19:23:02.349669 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:02.349238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k" event={"ID":"2e799479-db0f-470b-acf3-414803226167","Type":"ContainerStarted","Data":"133401a5b454963f96d971a5783669b63c529a75433cdc3544eacd049b4aba6f"}
Jul 22 19:23:02.349669 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:02.349298    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:02.367475 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:02.367429    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k" podStartSLOduration=1.003030009 podStartE2EDuration="1.367415034s" totalImagesPullingTime="364.385025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:23:02.366478942 +0000 UTC m=+2583.257019874" watchObservedRunningTime="2026-07-22 19:23:02.367415034 +0000 UTC m=+2583.257955965"
Jul 22 19:23:08.363099 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:08.363071    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:29.916682 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:29.916583    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"]
Jul 22 19:23:29.917308 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:29.916821    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx" podUID="7ca7fb2f-0145-41ac-93f4-9c4a9efb6231" containerName="maas-api" containerID="cri-o://874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288" gracePeriod=30
Jul 22 19:23:30.176255 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.176192    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:23:30.210849 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.210823    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vn8k7\" (UniqueName: \"kubernetes.io/projected/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-kube-api-access-vn8k7\") pod \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") "
Jul 22 19:23:30.210999 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.210853    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-maas-api-tls\") pod \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") "
Jul 22 19:23:30.210999 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.210878    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-openshift-service-ca\") pod \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\" (UID: \"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231\") "
Jul 22 19:23:30.211297 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.211267    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7ca7fb2f-0145-41ac-93f4-9c4a9efb6231" (UID: "7ca7fb2f-0145-41ac-93f4-9c4a9efb6231"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:23:30.212961 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.212926    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-kube-api-access-vn8k7" (OuterVolumeSpecName: "kube-api-access-vn8k7") pod "7ca7fb2f-0145-41ac-93f4-9c4a9efb6231" (UID: "7ca7fb2f-0145-41ac-93f4-9c4a9efb6231"). InnerVolumeSpecName "kube-api-access-vn8k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:23:30.213234 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.213202    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7ca7fb2f-0145-41ac-93f4-9c4a9efb6231" (UID: "7ca7fb2f-0145-41ac-93f4-9c4a9efb6231"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:23:30.311758 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.311719    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn8k7\" (UniqueName: \"kubernetes.io/projected/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-kube-api-access-vn8k7\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:23:30.311758 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.311754    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:23:30.311758 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.311764    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:23:30.488887 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.488803    2576 generic.go:356] "Generic (PLEG): container finished" podID="7ca7fb2f-0145-41ac-93f4-9c4a9efb6231" containerID="874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288" exitCode=0
Jul 22 19:23:30.488887 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.488843    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"
Jul 22 19:23:30.489069 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.488886    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx" event={"ID":"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231","Type":"ContainerDied","Data":"874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288"}
Jul 22 19:23:30.489069 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.488952    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx" event={"ID":"7ca7fb2f-0145-41ac-93f4-9c4a9efb6231","Type":"ContainerDied","Data":"9de7a7f9b7cb3467f4f19532f0f0c25b055276e0b9b9e85c3ed7f0f1a0933e3e"}
Jul 22 19:23:30.489069 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.488972    2576 scope.go:117] "RemoveContainer" containerID="874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288"
Jul 22 19:23:30.501430 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.501411    2576 scope.go:117] "RemoveContainer" containerID="874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288"
Jul 22 19:23:30.501671 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:23:30.501650    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288\": container with ID starting with 874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288 not found: ID does not exist" containerID="874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288"
Jul 22 19:23:30.501733 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.501679    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288"} err="failed to get container status \"874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288\": rpc error: code = NotFound desc = could not find container \"874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288\": container with ID starting with 874541aa06dc1d7985794f5c407cb14460765176c176fb089bdd82cc4aa35288 not found: ID does not exist"
Jul 22 19:23:30.511966 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.511942    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"]
Jul 22 19:23:30.515408 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:30.515368    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6c9526-75c887f949-6lrsx"]
Jul 22 19:23:31.688446 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:31.688405    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ca7fb2f-0145-41ac-93f4-9c4a9efb6231" path="/var/lib/kubelet/pods/7ca7fb2f-0145-41ac-93f4-9c4a9efb6231/volumes"
Jul 22 19:23:56.849976 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:56.849943    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"]
Jul 22 19:23:56.850703 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:56.850133    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k" podUID="2e799479-db0f-470b-acf3-414803226167" containerName="maas-api" containerID="cri-o://1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f" gracePeriod=30
Jul 22 19:23:57.098114 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.098085    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:57.244488 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.244453    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvkn4\" (UniqueName: \"kubernetes.io/projected/2e799479-db0f-470b-acf3-414803226167-kube-api-access-qvkn4\") pod \"2e799479-db0f-470b-acf3-414803226167\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") "
Jul 22 19:23:57.244701 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.244514    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e799479-db0f-470b-acf3-414803226167-maas-api-tls\") pod \"2e799479-db0f-470b-acf3-414803226167\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") "
Jul 22 19:23:57.244701 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.244576    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e799479-db0f-470b-acf3-414803226167-openshift-service-ca\") pod \"2e799479-db0f-470b-acf3-414803226167\" (UID: \"2e799479-db0f-470b-acf3-414803226167\") "
Jul 22 19:23:57.244996 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.244971    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e799479-db0f-470b-acf3-414803226167-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2e799479-db0f-470b-acf3-414803226167" (UID: "2e799479-db0f-470b-acf3-414803226167"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:23:57.246693 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.246669    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e799479-db0f-470b-acf3-414803226167-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2e799479-db0f-470b-acf3-414803226167" (UID: "2e799479-db0f-470b-acf3-414803226167"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:23:57.246693 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.246670    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e799479-db0f-470b-acf3-414803226167-kube-api-access-qvkn4" (OuterVolumeSpecName: "kube-api-access-qvkn4") pod "2e799479-db0f-470b-acf3-414803226167" (UID: "2e799479-db0f-470b-acf3-414803226167"). InnerVolumeSpecName "kube-api-access-qvkn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:23:57.345370 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.345335    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e799479-db0f-470b-acf3-414803226167-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:23:57.345370 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.345362    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e799479-db0f-470b-acf3-414803226167-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:23:57.345370 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.345371    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvkn4\" (UniqueName: \"kubernetes.io/projected/2e799479-db0f-470b-acf3-414803226167-kube-api-access-qvkn4\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:23:57.627587 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.627551    2576 generic.go:356] "Generic (PLEG): container finished" podID="2e799479-db0f-470b-acf3-414803226167" containerID="1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f" exitCode=0
Jul 22 19:23:57.627767 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.627599    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k" event={"ID":"2e799479-db0f-470b-acf3-414803226167","Type":"ContainerDied","Data":"1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f"}
Jul 22 19:23:57.627767 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.627607    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"
Jul 22 19:23:57.627767 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.627684    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k" event={"ID":"2e799479-db0f-470b-acf3-414803226167","Type":"ContainerDied","Data":"133401a5b454963f96d971a5783669b63c529a75433cdc3544eacd049b4aba6f"}
Jul 22 19:23:57.627767 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.627708    2576 scope.go:117] "RemoveContainer" containerID="1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f"
Jul 22 19:23:57.637807 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.637655    2576 scope.go:117] "RemoveContainer" containerID="1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f"
Jul 22 19:23:57.638079 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:23:57.638050    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f\": container with ID starting with 1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f not found: ID does not exist" containerID="1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f"
Jul 22 19:23:57.638156 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.638090    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f"} err="failed to get container status \"1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f\": rpc error: code = NotFound desc = could not find container \"1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f\": container with ID starting with 1d16bfe966c426d378db25a7144264fe85c8d6bd751fa5a67a180abb52cd963f not found: ID does not exist"
Jul 22 19:23:57.651469 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.651445    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"]
Jul 22 19:23:57.654415 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.654395    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-65c717-6bc5699788-wx49k"]
Jul 22 19:23:57.687845 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:23:57.687821    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e799479-db0f-470b-acf3-414803226167" path="/var/lib/kubelet/pods/2e799479-db0f-470b-acf3-414803226167/volumes"
Jul 22 19:24:11.885570 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.885532    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"]
Jul 22 19:24:11.886117 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.886095    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e799479-db0f-470b-acf3-414803226167" containerName="maas-api"
Jul 22 19:24:11.886195 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.886120    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e799479-db0f-470b-acf3-414803226167" containerName="maas-api"
Jul 22 19:24:11.886195 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.886159    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ca7fb2f-0145-41ac-93f4-9c4a9efb6231" containerName="maas-api"
Jul 22 19:24:11.886195 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.886168    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ca7fb2f-0145-41ac-93f4-9c4a9efb6231" containerName="maas-api"
Jul 22 19:24:11.886340 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.886266    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ca7fb2f-0145-41ac-93f4-9c4a9efb6231" containerName="maas-api"
Jul 22 19:24:11.886340 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.886279    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e799479-db0f-470b-acf3-414803226167" containerName="maas-api"
Jul 22 19:24:11.889132 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.889113    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:11.892015 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.891996    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-4ac498-serving-cert\""
Jul 22 19:24:11.897447 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.897388    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"]
Jul 22 19:24:11.969394 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.969365    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7b65f11-b82c-40c7-a27b-eb5185523a92-maas-api-tls\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:11.969547 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.969420    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7b65f11-b82c-40c7-a27b-eb5185523a92-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:11.969547 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:11.969514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnw2v\" (UniqueName: \"kubernetes.io/projected/b7b65f11-b82c-40c7-a27b-eb5185523a92-kube-api-access-gnw2v\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:12.070756 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:12.070723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7b65f11-b82c-40c7-a27b-eb5185523a92-maas-api-tls\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:12.070925 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:12.070777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7b65f11-b82c-40c7-a27b-eb5185523a92-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:12.070925 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:12.070831    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnw2v\" (UniqueName: \"kubernetes.io/projected/b7b65f11-b82c-40c7-a27b-eb5185523a92-kube-api-access-gnw2v\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:12.070925 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:24:12.070875    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-serving-cert: secret "maas-api-e2e-shared-a-4ac498-serving-cert" not found
Jul 22 19:24:12.071069 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:24:12.070944    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7b65f11-b82c-40c7-a27b-eb5185523a92-maas-api-tls podName:b7b65f11-b82c-40c7-a27b-eb5185523a92 nodeName:}" failed. No retries permitted until 2026-07-22 19:24:12.570927471 +0000 UTC m=+2653.461468382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7b65f11-b82c-40c7-a27b-eb5185523a92-maas-api-tls") pod "maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz" (UID: "b7b65f11-b82c-40c7-a27b-eb5185523a92") : secret "maas-api-e2e-shared-a-4ac498-serving-cert" not found
Jul 22 19:24:12.071488 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:12.071469    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7b65f11-b82c-40c7-a27b-eb5185523a92-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:12.081926 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:12.081898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnw2v\" (UniqueName: \"kubernetes.io/projected/b7b65f11-b82c-40c7-a27b-eb5185523a92-kube-api-access-gnw2v\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:12.576468 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:12.576437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7b65f11-b82c-40c7-a27b-eb5185523a92-maas-api-tls\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:12.578903 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:12.578869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7b65f11-b82c-40c7-a27b-eb5185523a92-maas-api-tls\") pod \"maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:12.800988 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:12.800956    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:12.939051 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:12.939025    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"]
Jul 22 19:24:12.941533 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:24:12.941500    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b65f11_b82c_40c7_a27b_eb5185523a92.slice/crio-a47e57499073b8f34c6f8883fe614b808c9aff3fe838bad778bf6fea5a15f11e WatchSource:0}: Error finding container a47e57499073b8f34c6f8883fe614b808c9aff3fe838bad778bf6fea5a15f11e: Status 404 returned error can't find the container with id a47e57499073b8f34c6f8883fe614b808c9aff3fe838bad778bf6fea5a15f11e
Jul 22 19:24:13.713750 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:13.713707    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz" event={"ID":"b7b65f11-b82c-40c7-a27b-eb5185523a92","Type":"ContainerStarted","Data":"a47e57499073b8f34c6f8883fe614b808c9aff3fe838bad778bf6fea5a15f11e"}
Jul 22 19:24:14.719510 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:14.719472    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz" event={"ID":"b7b65f11-b82c-40c7-a27b-eb5185523a92","Type":"ContainerStarted","Data":"4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6"}
Jul 22 19:24:14.719924 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:14.719579    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:14.738529 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:14.738479    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz" podStartSLOduration=2.83489063 podStartE2EDuration="3.738466937s" totalImagesPullingTime="903.576307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:24:14.737423364 +0000 UTC m=+2655.627964297" watchObservedRunningTime="2026-07-22 19:24:14.738466937 +0000 UTC m=+2655.629007871"
Jul 22 19:24:19.906150 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:19.906108    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"]
Jul 22 19:24:19.910066 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:19.910044    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:19.912813 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:19.912787    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-8f2692-serving-cert\""
Jul 22 19:24:19.921879 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:19.921854    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"]
Jul 22 19:24:20.042828 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.042795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3e50e0f-799a-443d-8013-7fe5d729e756-maas-api-tls\") pod \"maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.042999 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.042892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3e50e0f-799a-443d-8013-7fe5d729e756-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.042999 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.042917    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjfr\" (UniqueName: \"kubernetes.io/projected/a3e50e0f-799a-443d-8013-7fe5d729e756-kube-api-access-hdjfr\") pod \"maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.143554 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.143521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3e50e0f-799a-443d-8013-7fe5d729e756-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.143554 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.143559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjfr\" (UniqueName: \"kubernetes.io/projected/a3e50e0f-799a-443d-8013-7fe5d729e756-kube-api-access-hdjfr\") pod \"maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.143781 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.143646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3e50e0f-799a-443d-8013-7fe5d729e756-maas-api-tls\") pod \"maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.144313 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.144284    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3e50e0f-799a-443d-8013-7fe5d729e756-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.146147 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.146128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3e50e0f-799a-443d-8013-7fe5d729e756-maas-api-tls\") pod \"maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.151676 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.151653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjfr\" (UniqueName: \"kubernetes.io/projected/a3e50e0f-799a-443d-8013-7fe5d729e756-kube-api-access-hdjfr\") pod \"maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.223339 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.223283    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:20.354459 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.354431    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"]
Jul 22 19:24:20.356586 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:24:20.356554    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e50e0f_799a_443d_8013_7fe5d729e756.slice/crio-00bf7605e201e5f1ad1b31db34ac0a3fd1fb49bf68a5231a822ab7461eee2ddb WatchSource:0}: Error finding container 00bf7605e201e5f1ad1b31db34ac0a3fd1fb49bf68a5231a822ab7461eee2ddb: Status 404 returned error can't find the container with id 00bf7605e201e5f1ad1b31db34ac0a3fd1fb49bf68a5231a822ab7461eee2ddb
Jul 22 19:24:20.729914 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.729877    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:24:20.750322 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:20.750295    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w" event={"ID":"a3e50e0f-799a-443d-8013-7fe5d729e756","Type":"ContainerStarted","Data":"00bf7605e201e5f1ad1b31db34ac0a3fd1fb49bf68a5231a822ab7461eee2ddb"}
Jul 22 19:24:21.756065 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:21.756024    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w" event={"ID":"a3e50e0f-799a-443d-8013-7fe5d729e756","Type":"ContainerStarted","Data":"fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a"}
Jul 22 19:24:21.756493 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:21.756086    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:24:21.773119 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:21.773075    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w" podStartSLOduration=2.22829904 podStartE2EDuration="2.773059093s" totalImagesPullingTime="544.760053ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:24:21.771765827 +0000 UTC m=+2662.662306756" watchObservedRunningTime="2026-07-22 19:24:21.773059093 +0000 UTC m=+2662.663600025"
Jul 22 19:24:27.767171 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:24:27.767142    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:25:14.995826 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:14.995746    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"]
Jul 22 19:25:14.999012 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:14.998994    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.003120 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.003093    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8f2692\"/\"openshift-service-ca.crt\""
Jul 22 19:25:15.003244 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.003093    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8f2692\"/\"default-dockercfg-h442p\""
Jul 22 19:25:15.003411 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.003390    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8f2692\"/\"auth-test-model-8f2692-kserve-self-signed-certs\""
Jul 22 19:25:15.003545 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.003530    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8f2692\"/\"kube-root-ca.crt\""
Jul 22 19:25:15.009924 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.009903    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"]
Jul 22 19:25:15.103899 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.103864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-model-cache\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.104104 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.103912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-dshm\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.104104 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.103951    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-tmp-dir\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.104104 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.104048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k27jp\" (UniqueName: \"kubernetes.io/projected/303e6960-3e2f-42ed-95bf-7fab224d5c27-kube-api-access-k27jp\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.104276 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.104130    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/303e6960-3e2f-42ed-95bf-7fab224d5c27-tls-certs\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.104276 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.104176    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-home\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.205078 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.205044    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k27jp\" (UniqueName: \"kubernetes.io/projected/303e6960-3e2f-42ed-95bf-7fab224d5c27-kube-api-access-k27jp\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.205311 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.205085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/303e6960-3e2f-42ed-95bf-7fab224d5c27-tls-certs\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.205311 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.205106    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-home\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.205311 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.205154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-model-cache\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.205311 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.205181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-dshm\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.205311 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.205220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-tmp-dir\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.205621 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.205601    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-tmp-dir\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.205733 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.205688    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-home\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.205733 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.205718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-model-cache\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.207563 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.207537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-dshm\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.207724 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.207706    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/303e6960-3e2f-42ed-95bf-7fab224d5c27-tls-certs\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.212940 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.212921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k27jp\" (UniqueName: \"kubernetes.io/projected/303e6960-3e2f-42ed-95bf-7fab224d5c27-kube-api-access-k27jp\") pod \"auth-test-model-8f2692-kserve-545865cdb7-mpkj5\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") " pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.310043 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.309958    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:15.441410 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:15.441380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"]
Jul 22 19:25:15.443289 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:25:15.443264    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod303e6960_3e2f_42ed_95bf_7fab224d5c27.slice/crio-099de99ecbb669bcc14aa9685287bb074ab792dc8cef7c2da1bbc50e54ff07b0 WatchSource:0}: Error finding container 099de99ecbb669bcc14aa9685287bb074ab792dc8cef7c2da1bbc50e54ff07b0: Status 404 returned error can't find the container with id 099de99ecbb669bcc14aa9685287bb074ab792dc8cef7c2da1bbc50e54ff07b0
Jul 22 19:25:16.026168 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:16.026124    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5" event={"ID":"303e6960-3e2f-42ed-95bf-7fab224d5c27","Type":"ContainerStarted","Data":"6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654"}
Jul 22 19:25:16.026168 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:16.026166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5" event={"ID":"303e6960-3e2f-42ed-95bf-7fab224d5c27","Type":"ContainerStarted","Data":"099de99ecbb669bcc14aa9685287bb074ab792dc8cef7c2da1bbc50e54ff07b0"}
Jul 22 19:25:16.044562 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:16.044501    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5" podStartSLOduration=2.044486108 podStartE2EDuration="2.044486108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:25:16.043320379 +0000 UTC m=+2716.933861311" watchObservedRunningTime="2026-07-22 19:25:16.044486108 +0000 UTC m=+2716.935027039"
Jul 22 19:25:25.310771 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:25.310735    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:25.311359 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:25.310784    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:25.322948 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:25.322925    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:25:26.084151 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:25:26.084124    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:27:17.223997 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:17.223964    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"]
Jul 22 19:27:21.292007 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.291976    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"]
Jul 22 19:27:21.295200 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.295183    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.299246 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.299220    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4ac498\"/\"kube-root-ca.crt\""
Jul 22 19:27:21.299382 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.299267    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4ac498\"/\"sub-test-model-4ac498-kserve-self-signed-certs\""
Jul 22 19:27:21.299382 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.299226    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4ac498\"/\"openshift-service-ca.crt\""
Jul 22 19:27:21.299644 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.299608    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4ac498\"/\"default-dockercfg-gskw5\""
Jul 22 19:27:21.306379 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.306358    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"]
Jul 22 19:27:21.402593 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.402566    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tmp-dir\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.402753 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.402603    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t668x\" (UniqueName: \"kubernetes.io/projected/6a6a5554-e5ec-4560-93ff-e4f56151a49b-kube-api-access-t668x\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.402753 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.402696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-dshm\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.402830 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.402770    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-home\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.402830 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.402801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tls-certs\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.402899 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.402828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-model-cache\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.503492 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.503460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-home\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.503492 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.503496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tls-certs\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.503723 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.503520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-model-cache\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.503723 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.503543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tmp-dir\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.503723 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.503565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t668x\" (UniqueName: \"kubernetes.io/projected/6a6a5554-e5ec-4560-93ff-e4f56151a49b-kube-api-access-t668x\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.503723 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.503620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-dshm\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.503964 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.503939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-home\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.504026 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.503989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tmp-dir\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.504072 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.504050    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-model-cache\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.505812 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.505794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-dshm\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.506096 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.506079    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tls-certs\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.511415 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.511395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t668x\" (UniqueName: \"kubernetes.io/projected/6a6a5554-e5ec-4560-93ff-e4f56151a49b-kube-api-access-t668x\") pod \"sub-test-model-4ac498-kserve-9dd76766f-hlxqv\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") " pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.605395 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.605331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:21.833151 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:21.833123    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"]
Jul 22 19:27:21.848226 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:27:21.848200    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a6a5554_e5ec_4560_93ff_e4f56151a49b.slice/crio-636f8ba1b650695060934206e68e487b2ce74fc4f457b9b1ff71b609f684dc5d WatchSource:0}: Error finding container 636f8ba1b650695060934206e68e487b2ce74fc4f457b9b1ff71b609f684dc5d: Status 404 returned error can't find the container with id 636f8ba1b650695060934206e68e487b2ce74fc4f457b9b1ff71b609f684dc5d
Jul 22 19:27:22.657967 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:22.657926    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv" event={"ID":"6a6a5554-e5ec-4560-93ff-e4f56151a49b","Type":"ContainerStarted","Data":"737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d"}
Jul 22 19:27:22.658419 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:22.657983    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv" event={"ID":"6a6a5554-e5ec-4560-93ff-e4f56151a49b","Type":"ContainerStarted","Data":"636f8ba1b650695060934206e68e487b2ce74fc4f457b9b1ff71b609f684dc5d"}
Jul 22 19:27:22.687885 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:22.687840    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv" podStartSLOduration=1.687823531 podStartE2EDuration="1.687823531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:27:22.687300558 +0000 UTC m=+2843.577841490" watchObservedRunningTime="2026-07-22 19:27:22.687823531 +0000 UTC m=+2843.578364463"
Jul 22 19:27:31.606335 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:31.606295    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:31.606335 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:31.606341    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:31.620194 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:31.620173    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:31.711268 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:31.711242    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:27:32.245662 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.245584    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5" podUID="303e6960-3e2f-42ed-95bf-7fab224d5c27" containerName="main" containerID="cri-o://6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654" gracePeriod=45
Jul 22 19:27:32.505011 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.504957    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:27:32.600517 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.600488    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-tmp-dir\") pod \"303e6960-3e2f-42ed-95bf-7fab224d5c27\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") "
Jul 22 19:27:32.600723 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.600609    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-model-cache\") pod \"303e6960-3e2f-42ed-95bf-7fab224d5c27\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") "
Jul 22 19:27:32.600723 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.600714    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-dshm\") pod \"303e6960-3e2f-42ed-95bf-7fab224d5c27\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") "
Jul 22 19:27:32.600866 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.600757    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k27jp\" (UniqueName: \"kubernetes.io/projected/303e6960-3e2f-42ed-95bf-7fab224d5c27-kube-api-access-k27jp\") pod \"303e6960-3e2f-42ed-95bf-7fab224d5c27\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") "
Jul 22 19:27:32.600866 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.600755    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "303e6960-3e2f-42ed-95bf-7fab224d5c27" (UID: "303e6960-3e2f-42ed-95bf-7fab224d5c27"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:27:32.600866 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.600788    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-home\") pod \"303e6960-3e2f-42ed-95bf-7fab224d5c27\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") "
Jul 22 19:27:32.600866 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.600830    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-model-cache" (OuterVolumeSpecName: "model-cache") pod "303e6960-3e2f-42ed-95bf-7fab224d5c27" (UID: "303e6960-3e2f-42ed-95bf-7fab224d5c27"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:27:32.601070 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.600875    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/303e6960-3e2f-42ed-95bf-7fab224d5c27-tls-certs\") pod \"303e6960-3e2f-42ed-95bf-7fab224d5c27\" (UID: \"303e6960-3e2f-42ed-95bf-7fab224d5c27\") "
Jul 22 19:27:32.601070 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.601005    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-home" (OuterVolumeSpecName: "home") pod "303e6960-3e2f-42ed-95bf-7fab224d5c27" (UID: "303e6960-3e2f-42ed-95bf-7fab224d5c27"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:27:32.601248 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.601222    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-model-cache\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:27:32.601329 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.601257    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-home\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:27:32.601329 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.601266    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-tmp-dir\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:27:32.602874 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.602852    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-dshm" (OuterVolumeSpecName: "dshm") pod "303e6960-3e2f-42ed-95bf-7fab224d5c27" (UID: "303e6960-3e2f-42ed-95bf-7fab224d5c27"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:27:32.602987 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.602953    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/303e6960-3e2f-42ed-95bf-7fab224d5c27-kube-api-access-k27jp" (OuterVolumeSpecName: "kube-api-access-k27jp") pod "303e6960-3e2f-42ed-95bf-7fab224d5c27" (UID: "303e6960-3e2f-42ed-95bf-7fab224d5c27"). InnerVolumeSpecName "kube-api-access-k27jp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:27:32.603032 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.603007    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/303e6960-3e2f-42ed-95bf-7fab224d5c27-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "303e6960-3e2f-42ed-95bf-7fab224d5c27" (UID: "303e6960-3e2f-42ed-95bf-7fab224d5c27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:27:32.702079 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.702053    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/303e6960-3e2f-42ed-95bf-7fab224d5c27-dshm\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:27:32.702079 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.702077    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k27jp\" (UniqueName: \"kubernetes.io/projected/303e6960-3e2f-42ed-95bf-7fab224d5c27-kube-api-access-k27jp\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:27:32.702429 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.702089    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/303e6960-3e2f-42ed-95bf-7fab224d5c27-tls-certs\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:27:32.706234 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.706206    2576 generic.go:356] "Generic (PLEG): container finished" podID="303e6960-3e2f-42ed-95bf-7fab224d5c27" containerID="6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654" exitCode=0
Jul 22 19:27:32.706343 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.706249    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"
Jul 22 19:27:32.706343 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.706298    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5" event={"ID":"303e6960-3e2f-42ed-95bf-7fab224d5c27","Type":"ContainerDied","Data":"6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654"}
Jul 22 19:27:32.706343 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.706337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5" event={"ID":"303e6960-3e2f-42ed-95bf-7fab224d5c27","Type":"ContainerDied","Data":"099de99ecbb669bcc14aa9685287bb074ab792dc8cef7c2da1bbc50e54ff07b0"}
Jul 22 19:27:32.706467 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.706358    2576 scope.go:117] "RemoveContainer" containerID="6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654"
Jul 22 19:27:32.722080 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.722063    2576 scope.go:117] "RemoveContainer" containerID="6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654"
Jul 22 19:27:32.722356 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:27:32.722335    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654\": container with ID starting with 6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654 not found: ID does not exist" containerID="6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654"
Jul 22 19:27:32.722403 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.722365    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654"} err="failed to get container status \"6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654\": rpc error: code = NotFound desc = could not find container \"6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654\": container with ID starting with 6ef74923d1be01d9995b1c95090222f93a1adf4bf3ddbf058f507c87ac3d0654 not found: ID does not exist"
Jul 22 19:27:32.735980 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.735958    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"]
Jul 22 19:27:32.743238 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:32.743218    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8f2692/auth-test-model-8f2692-kserve-545865cdb7-mpkj5"]
Jul 22 19:27:33.688717 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:27:33.688682    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="303e6960-3e2f-42ed-95bf-7fab224d5c27" path="/var/lib/kubelet/pods/303e6960-3e2f-42ed-95bf-7fab224d5c27/volumes"
Jul 22 19:28:56.590771 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:28:56.590726    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"]
Jul 22 19:28:59.921544 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:28:59.921444    2576 scope.go:117] "RemoveContainer" containerID="5e3a9d1623dc9ac85850c6b22cc0495433e9c2e0d982ecc986fa6cb5595955d3"
Jul 22 19:29:11.611123 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.611079    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv" podUID="6a6a5554-e5ec-4560-93ff-e4f56151a49b" containerName="main" containerID="cri-o://737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d" gracePeriod=45
Jul 22 19:29:11.701310 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.701279    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv" podUID="6a6a5554-e5ec-4560-93ff-e4f56151a49b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": dial tcp 10.134.0.71:8000: connect: connection refused"
Jul 22 19:29:11.873392 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.873368    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:29:11.957882 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.957850    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tmp-dir\") pod \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") "
Jul 22 19:29:11.958056 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.957909    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t668x\" (UniqueName: \"kubernetes.io/projected/6a6a5554-e5ec-4560-93ff-e4f56151a49b-kube-api-access-t668x\") pod \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") "
Jul 22 19:29:11.958056 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.957968    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-home\") pod \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") "
Jul 22 19:29:11.958056 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.957994    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-model-cache\") pod \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") "
Jul 22 19:29:11.958056 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.958014    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tls-certs\") pod \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") "
Jul 22 19:29:11.958056 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.958036    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-dshm\") pod \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\" (UID: \"6a6a5554-e5ec-4560-93ff-e4f56151a49b\") "
Jul 22 19:29:11.958384 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.958091    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6a6a5554-e5ec-4560-93ff-e4f56151a49b" (UID: "6a6a5554-e5ec-4560-93ff-e4f56151a49b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:29:11.958384 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.958245    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-home" (OuterVolumeSpecName: "home") pod "6a6a5554-e5ec-4560-93ff-e4f56151a49b" (UID: "6a6a5554-e5ec-4560-93ff-e4f56151a49b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:29:11.958384 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.958269    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-model-cache" (OuterVolumeSpecName: "model-cache") pod "6a6a5554-e5ec-4560-93ff-e4f56151a49b" (UID: "6a6a5554-e5ec-4560-93ff-e4f56151a49b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:29:11.958384 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.958323    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tmp-dir\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:29:11.960316 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.960290    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6a6a5554-e5ec-4560-93ff-e4f56151a49b" (UID: "6a6a5554-e5ec-4560-93ff-e4f56151a49b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:29:11.960415 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.960312    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a6a5554-e5ec-4560-93ff-e4f56151a49b-kube-api-access-t668x" (OuterVolumeSpecName: "kube-api-access-t668x") pod "6a6a5554-e5ec-4560-93ff-e4f56151a49b" (UID: "6a6a5554-e5ec-4560-93ff-e4f56151a49b"). InnerVolumeSpecName "kube-api-access-t668x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:29:11.960415 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:11.960334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-dshm" (OuterVolumeSpecName: "dshm") pod "6a6a5554-e5ec-4560-93ff-e4f56151a49b" (UID: "6a6a5554-e5ec-4560-93ff-e4f56151a49b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:29:12.059042 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.059014    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-model-cache\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:29:12.059042 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.059037    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a6a5554-e5ec-4560-93ff-e4f56151a49b-tls-certs\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:29:12.059042 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.059046    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-dshm\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:29:12.059231 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.059054    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t668x\" (UniqueName: \"kubernetes.io/projected/6a6a5554-e5ec-4560-93ff-e4f56151a49b-kube-api-access-t668x\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:29:12.059231 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.059064    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a6a5554-e5ec-4560-93ff-e4f56151a49b-home\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:29:12.189861 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.189774    2576 generic.go:356] "Generic (PLEG): container finished" podID="6a6a5554-e5ec-4560-93ff-e4f56151a49b" containerID="737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d" exitCode=0
Jul 22 19:29:12.189861 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.189818    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"
Jul 22 19:29:12.190050 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.189821    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv" event={"ID":"6a6a5554-e5ec-4560-93ff-e4f56151a49b","Type":"ContainerDied","Data":"737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d"}
Jul 22 19:29:12.190050 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.189910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv" event={"ID":"6a6a5554-e5ec-4560-93ff-e4f56151a49b","Type":"ContainerDied","Data":"636f8ba1b650695060934206e68e487b2ce74fc4f457b9b1ff71b609f684dc5d"}
Jul 22 19:29:12.190050 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.189927    2576 scope.go:117] "RemoveContainer" containerID="737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d"
Jul 22 19:29:12.198813 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.198781    2576 scope.go:117] "RemoveContainer" containerID="737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d"
Jul 22 19:29:12.199064 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:29:12.199044    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d\": container with ID starting with 737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d not found: ID does not exist" containerID="737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d"
Jul 22 19:29:12.199143 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.199078    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d"} err="failed to get container status \"737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d\": rpc error: code = NotFound desc = could not find container \"737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d\": container with ID starting with 737b46ac3552b321eb6920d5389723c67a3ef7b47551c178b7abb9bc420af91d not found: ID does not exist"
Jul 22 19:29:12.211165 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.211140    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"]
Jul 22 19:29:12.216334 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:12.216312    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/sub-test-model-4ac498-kserve-9dd76766f-hlxqv"]
Jul 22 19:29:13.687771 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:13.687737    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a6a5554-e5ec-4560-93ff-e4f56151a49b" path="/var/lib/kubelet/pods/6a6a5554-e5ec-4560-93ff-e4f56151a49b/volumes"
Jul 22 19:29:17.592304 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.592266    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"]
Jul 22 19:29:17.592718 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.592677    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="303e6960-3e2f-42ed-95bf-7fab224d5c27" containerName="main"
Jul 22 19:29:17.592718 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.592689    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="303e6960-3e2f-42ed-95bf-7fab224d5c27" containerName="main"
Jul 22 19:29:17.592718 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.592712    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a6a5554-e5ec-4560-93ff-e4f56151a49b" containerName="main"
Jul 22 19:29:17.592718 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.592718    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a6a5554-e5ec-4560-93ff-e4f56151a49b" containerName="main"
Jul 22 19:29:17.592857 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.592779    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a6a5554-e5ec-4560-93ff-e4f56151a49b" containerName="main"
Jul 22 19:29:17.592857 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.592786    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="303e6960-3e2f-42ed-95bf-7fab224d5c27" containerName="main"
Jul 22 19:29:17.596962 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.596941    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.600875 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.600851    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4ac498\"/\"kube-root-ca.crt\""
Jul 22 19:29:17.601012 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.600851    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4ac498\"/\"default-dockercfg-gskw5\""
Jul 22 19:29:17.601012 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.600889    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4ac498\"/\"rate-test-model-4ac498-kserve-self-signed-certs\""
Jul 22 19:29:17.601012 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.600851    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4ac498\"/\"openshift-service-ca.crt\""
Jul 22 19:29:17.603828 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.603805    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"]
Jul 22 19:29:17.704866 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.704838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-home\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.705059 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.704897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99676424-68b7-4256-990e-734e5c3ba73b-tls-certs\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.705059 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.704955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-tmp-dir\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.705059 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.704990    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlf8r\" (UniqueName: \"kubernetes.io/projected/99676424-68b7-4256-990e-734e5c3ba73b-kube-api-access-zlf8r\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.705221 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.705067    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-dshm\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.705221 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.705093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-model-cache\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.806003 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.805967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-dshm\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.806003 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.806002    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-model-cache\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.806261 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.806030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-home\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.806261 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.806086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99676424-68b7-4256-990e-734e5c3ba73b-tls-certs\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.806261 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.806123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-tmp-dir\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.806261 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.806147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlf8r\" (UniqueName: \"kubernetes.io/projected/99676424-68b7-4256-990e-734e5c3ba73b-kube-api-access-zlf8r\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.806533 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.806507    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-model-cache\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.806574 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.806528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-tmp-dir\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.806574 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.806506    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-home\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.808260 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.808239    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-dshm\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.808608 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.808589    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99676424-68b7-4256-990e-734e5c3ba73b-tls-certs\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.813882 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.813859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlf8r\" (UniqueName: \"kubernetes.io/projected/99676424-68b7-4256-990e-734e5c3ba73b-kube-api-access-zlf8r\") pod \"rate-test-model-4ac498-kserve-78754f59f9-4bt46\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") " pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:17.906739 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:17.906713    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:18.097468 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:18.097430    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"]
Jul 22 19:29:18.115975 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:29:18.115942    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99676424_68b7_4256_990e_734e5c3ba73b.slice/crio-8f55fd28f33823b13093daaf1293566d63f166a5aa316a7c4e655e8707b64b1d WatchSource:0}: Error finding container 8f55fd28f33823b13093daaf1293566d63f166a5aa316a7c4e655e8707b64b1d: Status 404 returned error can't find the container with id 8f55fd28f33823b13093daaf1293566d63f166a5aa316a7c4e655e8707b64b1d
Jul 22 19:29:18.226831 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:18.226793    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46" event={"ID":"99676424-68b7-4256-990e-734e5c3ba73b","Type":"ContainerStarted","Data":"d042ffb8cc318e9fb1f9c344820b4e2ac4d45fd9d33a92b57ad5b304cb41b192"}
Jul 22 19:29:18.226975 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:18.226840    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46" event={"ID":"99676424-68b7-4256-990e-734e5c3ba73b","Type":"ContainerStarted","Data":"8f55fd28f33823b13093daaf1293566d63f166a5aa316a7c4e655e8707b64b1d"}
Jul 22 19:29:18.245744 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:18.245691    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46" podStartSLOduration=1.245671576 podStartE2EDuration="1.245671576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:29:18.242554789 +0000 UTC m=+2959.133095722" watchObservedRunningTime="2026-07-22 19:29:18.245671576 +0000 UTC m=+2959.136212510"
Jul 22 19:29:27.907680 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:27.907646    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:27.908263 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:27.907711    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:27.920215 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:27.920194    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:29:28.290691 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:29:28.290603    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:30:00.131866 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.131827    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq"]
Jul 22 19:30:00.136246 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.136228    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz"]
Jul 22 19:30:00.136360 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.136341    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq"
Jul 22 19:30:00.139039 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.139022    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5ttk\""
Jul 22 19:30:00.139929 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.139907    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv"]
Jul 22 19:30:00.140065 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.140023    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz"
Jul 22 19:30:00.143344 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.143323    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq"]
Jul 22 19:30:00.143584 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.143531    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv"
Jul 22 19:30:00.145669 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.145602    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz"]
Jul 22 19:30:00.148840 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.148164    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv"]
Jul 22 19:30:00.151708 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.151687    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq"
Jul 22 19:30:00.159881 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.159865    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz"
Jul 22 19:30:00.166914 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.166887    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv"
Jul 22 19:30:00.367125 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.367094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq"]
Jul 22 19:30:00.380134 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:30:00.380100    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca8f4f9_e0e7_4fe1_9fd5_394c3a7734a8.slice/crio-a78aad54ab99e0f73dd0edf3c79b1c03ff36dead4b74098c12014f1caf473599 WatchSource:0}: Error finding container a78aad54ab99e0f73dd0edf3c79b1c03ff36dead4b74098c12014f1caf473599: Status 404 returned error can't find the container with id a78aad54ab99e0f73dd0edf3c79b1c03ff36dead4b74098c12014f1caf473599
Jul 22 19:30:00.437022 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.436986    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq" event={"ID":"1ca8f4f9-e0e7-4fe1-9fd5-394c3a7734a8","Type":"ContainerStarted","Data":"a78aad54ab99e0f73dd0edf3c79b1c03ff36dead4b74098c12014f1caf473599"}
Jul 22 19:30:00.595586 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.595562    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz"]
Jul 22 19:30:00.617077 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:30:00.617057    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c373e9_2a6f_4dca_88ee_f98af78a0338.slice/crio-d98fa28387a2f482b269f809d9de09f3e93cf93f6d57f14f27f2278d597fb49f WatchSource:0}: Error finding container d98fa28387a2f482b269f809d9de09f3e93cf93f6d57f14f27f2278d597fb49f: Status 404 returned error can't find the container with id d98fa28387a2f482b269f809d9de09f3e93cf93f6d57f14f27f2278d597fb49f
Jul 22 19:30:00.631386 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:00.631362    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv"]
Jul 22 19:30:00.642575 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:30:00.642553    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd98480_269d_4eb9_a96b_80de3aab98b0.slice/crio-6d5e59f50b6254e05f3a9d445cec159e2e4418443ea648fcdd9b424c5b6aca87 WatchSource:0}: Error finding container 6d5e59f50b6254e05f3a9d445cec159e2e4418443ea648fcdd9b424c5b6aca87: Status 404 returned error can't find the container with id 6d5e59f50b6254e05f3a9d445cec159e2e4418443ea648fcdd9b424c5b6aca87
Jul 22 19:30:01.443719 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:01.443681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz" event={"ID":"91c373e9-2a6f-4dca-88ee-f98af78a0338","Type":"ContainerStarted","Data":"9fa93070246efb0d62d64d3d4a51994de4fc5467aaf9eb297f843941c75ddb9c"}
Jul 22 19:30:01.444102 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:01.443748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz" event={"ID":"91c373e9-2a6f-4dca-88ee-f98af78a0338","Type":"ContainerStarted","Data":"d98fa28387a2f482b269f809d9de09f3e93cf93f6d57f14f27f2278d597fb49f"}
Jul 22 19:30:01.445270 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:01.445239    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv" event={"ID":"5cd98480-269d-4eb9-a96b-80de3aab98b0","Type":"ContainerStarted","Data":"90b2b2015518c4ca138f1bc69c497f7fd2adc683295d8900d88bf3ef95dd36d2"}
Jul 22 19:30:01.445388 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:01.445290    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv" event={"ID":"5cd98480-269d-4eb9-a96b-80de3aab98b0","Type":"ContainerStarted","Data":"6d5e59f50b6254e05f3a9d445cec159e2e4418443ea648fcdd9b424c5b6aca87"}
Jul 22 19:30:01.446585 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:01.446567    2576 generic.go:356] "Generic (PLEG): container finished" podID="1ca8f4f9-e0e7-4fe1-9fd5-394c3a7734a8" containerID="00d7c649c7363cdedd64f4449de14f8b462d5e891482265082fb877c174c1e77" exitCode=0
Jul 22 19:30:01.446674 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:01.446646    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq" event={"ID":"1ca8f4f9-e0e7-4fe1-9fd5-394c3a7734a8","Type":"ContainerDied","Data":"00d7c649c7363cdedd64f4449de14f8b462d5e891482265082fb877c174c1e77"}
Jul 22 19:30:01.460008 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:01.459966    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz" podStartSLOduration=1.459953042 podStartE2EDuration="1.459953042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:30:01.45808092 +0000 UTC m=+3002.348621859" watchObservedRunningTime="2026-07-22 19:30:01.459953042 +0000 UTC m=+3002.350493974"
Jul 22 19:30:01.474269 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:01.474233    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv" podStartSLOduration=1.474220673 podStartE2EDuration="1.474220673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:30:01.47297095 +0000 UTC m=+3002.363511883" watchObservedRunningTime="2026-07-22 19:30:01.474220673 +0000 UTC m=+3002.364761604"
Jul 22 19:30:02.453685 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:02.453624    2576 generic.go:356] "Generic (PLEG): container finished" podID="91c373e9-2a6f-4dca-88ee-f98af78a0338" containerID="9fa93070246efb0d62d64d3d4a51994de4fc5467aaf9eb297f843941c75ddb9c" exitCode=0
Jul 22 19:30:02.454142 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:02.453735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz" event={"ID":"91c373e9-2a6f-4dca-88ee-f98af78a0338","Type":"ContainerDied","Data":"9fa93070246efb0d62d64d3d4a51994de4fc5467aaf9eb297f843941c75ddb9c"}
Jul 22 19:30:02.455128 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:02.455105    2576 generic.go:356] "Generic (PLEG): container finished" podID="5cd98480-269d-4eb9-a96b-80de3aab98b0" containerID="90b2b2015518c4ca138f1bc69c497f7fd2adc683295d8900d88bf3ef95dd36d2" exitCode=0
Jul 22 19:30:02.455238 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:02.455163    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv" event={"ID":"5cd98480-269d-4eb9-a96b-80de3aab98b0","Type":"ContainerDied","Data":"90b2b2015518c4ca138f1bc69c497f7fd2adc683295d8900d88bf3ef95dd36d2"}
Jul 22 19:30:02.586136 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:02.586111    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq"
Jul 22 19:30:03.460795 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:03.460763    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq"
Jul 22 19:30:03.461180 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:03.460795    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-xn5bq" event={"ID":"1ca8f4f9-e0e7-4fe1-9fd5-394c3a7734a8","Type":"ContainerDied","Data":"a78aad54ab99e0f73dd0edf3c79b1c03ff36dead4b74098c12014f1caf473599"}
Jul 22 19:30:03.461180 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:03.460847    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a78aad54ab99e0f73dd0edf3c79b1c03ff36dead4b74098c12014f1caf473599"
Jul 22 19:30:03.605383 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:03.605363    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv"
Jul 22 19:30:03.608345 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:03.608328    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz"
Jul 22 19:30:04.466767 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:04.466740    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz"
Jul 22 19:30:04.467160 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:04.466744    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz" event={"ID":"91c373e9-2a6f-4dca-88ee-f98af78a0338","Type":"ContainerDied","Data":"d98fa28387a2f482b269f809d9de09f3e93cf93f6d57f14f27f2278d597fb49f"}
Jul 22 19:30:04.467160 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:04.466868    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d98fa28387a2f482b269f809d9de09f3e93cf93f6d57f14f27f2278d597fb49f"
Jul 22 19:30:04.468076 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:04.468056    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv"
Jul 22 19:30:04.468195 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:04.468099    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv" event={"ID":"5cd98480-269d-4eb9-a96b-80de3aab98b0","Type":"ContainerDied","Data":"6d5e59f50b6254e05f3a9d445cec159e2e4418443ea648fcdd9b424c5b6aca87"}
Jul 22 19:30:04.468195 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:04.468138    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d5e59f50b6254e05f3a9d445cec159e2e4418443ea648fcdd9b424c5b6aca87"
Jul 22 19:30:26.419822 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:26.419787    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"]
Jul 22 19:30:28.480145 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480110    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7789-140041791286960"]
Jul 22 19:30:28.480509 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480462    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ca8f4f9-e0e7-4fe1-9fd5-394c3a7734a8" containerName="cleanup"
Jul 22 19:30:28.480509 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480473    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ca8f4f9-e0e7-4fe1-9fd5-394c3a7734a8" containerName="cleanup"
Jul 22 19:30:28.480509 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480489    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cd98480-269d-4eb9-a96b-80de3aab98b0" containerName="cleanup"
Jul 22 19:30:28.480509 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480495    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cd98480-269d-4eb9-a96b-80de3aab98b0" containerName="cleanup"
Jul 22 19:30:28.480700 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480522    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91c373e9-2a6f-4dca-88ee-f98af78a0338" containerName="cleanup"
Jul 22 19:30:28.480700 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480529    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="91c373e9-2a6f-4dca-88ee-f98af78a0338" containerName="cleanup"
Jul 22 19:30:28.480700 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480590    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ca8f4f9-e0e7-4fe1-9fd5-394c3a7734a8" containerName="cleanup"
Jul 22 19:30:28.480700 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480600    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="91c373e9-2a6f-4dca-88ee-f98af78a0338" containerName="cleanup"
Jul 22 19:30:28.480700 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.480609    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cd98480-269d-4eb9-a96b-80de3aab98b0" containerName="cleanup"
Jul 22 19:30:28.483818 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.483797    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7789-140041791286960"
Jul 22 19:30:28.486217 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.486197    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m4ctx\""
Jul 22 19:30:28.489524 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.489502    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7789-140041791286960"]
Jul 22 19:30:28.515759 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.515736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2ds6\" (UniqueName: \"kubernetes.io/projected/83a8e0cf-64b1-443d-a6d0-666aa1bfff11-kube-api-access-n2ds6\") pod \"test-curl-7789-140041791286960\" (UID: \"83a8e0cf-64b1-443d-a6d0-666aa1bfff11\") " pod="opendatahub/test-curl-7789-140041791286960"
Jul 22 19:30:28.616391 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.616364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2ds6\" (UniqueName: \"kubernetes.io/projected/83a8e0cf-64b1-443d-a6d0-666aa1bfff11-kube-api-access-n2ds6\") pod \"test-curl-7789-140041791286960\" (UID: \"83a8e0cf-64b1-443d-a6d0-666aa1bfff11\") " pod="opendatahub/test-curl-7789-140041791286960"
Jul 22 19:30:28.624742 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.624716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2ds6\" (UniqueName: \"kubernetes.io/projected/83a8e0cf-64b1-443d-a6d0-666aa1bfff11-kube-api-access-n2ds6\") pod \"test-curl-7789-140041791286960\" (UID: \"83a8e0cf-64b1-443d-a6d0-666aa1bfff11\") " pod="opendatahub/test-curl-7789-140041791286960"
Jul 22 19:30:28.795403 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.795313    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7789-140041791286960"
Jul 22 19:30:28.947871 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.947839    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7789-140041791286960"]
Jul 22 19:30:28.952592 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:30:28.952567    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83a8e0cf_64b1_443d_a6d0_666aa1bfff11.slice/crio-4837a0517b1317856357f3ffe5333d192861a57cb1830d648eac800943d53dee WatchSource:0}: Error finding container 4837a0517b1317856357f3ffe5333d192861a57cb1830d648eac800943d53dee: Status 404 returned error can't find the container with id 4837a0517b1317856357f3ffe5333d192861a57cb1830d648eac800943d53dee
Jul 22 19:30:28.953887 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:28.953870    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 19:30:29.598867 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:29.598826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7789-140041791286960" event={"ID":"83a8e0cf-64b1-443d-a6d0-666aa1bfff11","Type":"ContainerStarted","Data":"4837a0517b1317856357f3ffe5333d192861a57cb1830d648eac800943d53dee"}
Jul 22 19:30:30.605894 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:30.605851    2576 generic.go:356] "Generic (PLEG): container finished" podID="83a8e0cf-64b1-443d-a6d0-666aa1bfff11" containerID="6819d6092ee0f6df4a9330800b0b0c23c2cf85a5f5415ec08feaccfb0253086a" exitCode=0
Jul 22 19:30:30.606292 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:30.605914    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7789-140041791286960" event={"ID":"83a8e0cf-64b1-443d-a6d0-666aa1bfff11","Type":"ContainerDied","Data":"6819d6092ee0f6df4a9330800b0b0c23c2cf85a5f5415ec08feaccfb0253086a"}
Jul 22 19:30:31.747334 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:31.747308    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7789-140041791286960"
Jul 22 19:30:31.781472 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:31.781449    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7789-140041791286960_83a8e0cf-64b1-443d-a6d0-666aa1bfff11/test-curl-7789-140041791286960/0.log"
Jul 22 19:30:31.807326 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:31.807302    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7789-140041791286960"]
Jul 22 19:30:31.813456 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:31.813435    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7789-140041791286960"]
Jul 22 19:30:31.846826 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:31.846804    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2ds6\" (UniqueName: \"kubernetes.io/projected/83a8e0cf-64b1-443d-a6d0-666aa1bfff11-kube-api-access-n2ds6\") pod \"83a8e0cf-64b1-443d-a6d0-666aa1bfff11\" (UID: \"83a8e0cf-64b1-443d-a6d0-666aa1bfff11\") "
Jul 22 19:30:31.848886 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:31.848856    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a8e0cf-64b1-443d-a6d0-666aa1bfff11-kube-api-access-n2ds6" (OuterVolumeSpecName: "kube-api-access-n2ds6") pod "83a8e0cf-64b1-443d-a6d0-666aa1bfff11" (UID: "83a8e0cf-64b1-443d-a6d0-666aa1bfff11"). InnerVolumeSpecName "kube-api-access-n2ds6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:30:31.947738 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:31.947709    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2ds6\" (UniqueName: \"kubernetes.io/projected/83a8e0cf-64b1-443d-a6d0-666aa1bfff11-kube-api-access-n2ds6\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:32.617662 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:32.617610    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7789-140041791286960"
Jul 22 19:30:32.617662 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:32.617663    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4837a0517b1317856357f3ffe5333d192861a57cb1830d648eac800943d53dee"
Jul 22 19:30:32.627818 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:30:32.627793    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7789-140041791286960\" is forbidden: User \"system:node:ip-10-0-139-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-185.ec2.internal' and this object" podUID="83a8e0cf-64b1-443d-a6d0-666aa1bfff11" pod="opendatahub/test-curl-7789-140041791286960"
Jul 22 19:30:33.689218 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:33.689181    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a8e0cf-64b1-443d-a6d0-666aa1bfff11" path="/var/lib/kubelet/pods/83a8e0cf-64b1-443d-a6d0-666aa1bfff11/volumes"
Jul 22 19:30:41.446454 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.446389    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46" podUID="99676424-68b7-4256-990e-734e5c3ba73b" containerName="main" containerID="cri-o://d042ffb8cc318e9fb1f9c344820b4e2ac4d45fd9d33a92b57ad5b304cb41b192" gracePeriod=45
Jul 22 19:30:41.674565 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.674535    2576 generic.go:356] "Generic (PLEG): container finished" podID="99676424-68b7-4256-990e-734e5c3ba73b" containerID="d042ffb8cc318e9fb1f9c344820b4e2ac4d45fd9d33a92b57ad5b304cb41b192" exitCode=0
Jul 22 19:30:41.674743 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.674613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46" event={"ID":"99676424-68b7-4256-990e-734e5c3ba73b","Type":"ContainerDied","Data":"d042ffb8cc318e9fb1f9c344820b4e2ac4d45fd9d33a92b57ad5b304cb41b192"}
Jul 22 19:30:41.699586 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.699537    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:30:41.833004 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.832976    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99676424-68b7-4256-990e-734e5c3ba73b-tls-certs\") pod \"99676424-68b7-4256-990e-734e5c3ba73b\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") "
Jul 22 19:30:41.833188 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.833017    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-home\") pod \"99676424-68b7-4256-990e-734e5c3ba73b\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") "
Jul 22 19:30:41.833188 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.833045    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-model-cache\") pod \"99676424-68b7-4256-990e-734e5c3ba73b\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") "
Jul 22 19:30:41.833188 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.833099    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlf8r\" (UniqueName: \"kubernetes.io/projected/99676424-68b7-4256-990e-734e5c3ba73b-kube-api-access-zlf8r\") pod \"99676424-68b7-4256-990e-734e5c3ba73b\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") "
Jul 22 19:30:41.833188 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.833144    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-tmp-dir\") pod \"99676424-68b7-4256-990e-734e5c3ba73b\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") "
Jul 22 19:30:41.833421 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.833201    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-dshm\") pod \"99676424-68b7-4256-990e-734e5c3ba73b\" (UID: \"99676424-68b7-4256-990e-734e5c3ba73b\") "
Jul 22 19:30:41.833421 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.833326    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-home" (OuterVolumeSpecName: "home") pod "99676424-68b7-4256-990e-734e5c3ba73b" (UID: "99676424-68b7-4256-990e-734e5c3ba73b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:30:41.833530 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.833468    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-model-cache" (OuterVolumeSpecName: "model-cache") pod "99676424-68b7-4256-990e-734e5c3ba73b" (UID: "99676424-68b7-4256-990e-734e5c3ba73b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:30:41.833588 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.833525    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-home\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:41.833588 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.833561    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "99676424-68b7-4256-990e-734e5c3ba73b" (UID: "99676424-68b7-4256-990e-734e5c3ba73b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:30:41.835467 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.835431    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99676424-68b7-4256-990e-734e5c3ba73b-kube-api-access-zlf8r" (OuterVolumeSpecName: "kube-api-access-zlf8r") pod "99676424-68b7-4256-990e-734e5c3ba73b" (UID: "99676424-68b7-4256-990e-734e5c3ba73b"). InnerVolumeSpecName "kube-api-access-zlf8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:30:41.835595 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.835551    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99676424-68b7-4256-990e-734e5c3ba73b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "99676424-68b7-4256-990e-734e5c3ba73b" (UID: "99676424-68b7-4256-990e-734e5c3ba73b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:30:41.835739 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.835721    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-dshm" (OuterVolumeSpecName: "dshm") pod "99676424-68b7-4256-990e-734e5c3ba73b" (UID: "99676424-68b7-4256-990e-734e5c3ba73b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:30:41.934838 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.934798    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99676424-68b7-4256-990e-734e5c3ba73b-tls-certs\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:41.934838 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.934827    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-model-cache\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:41.934838 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.934837    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlf8r\" (UniqueName: \"kubernetes.io/projected/99676424-68b7-4256-990e-734e5c3ba73b-kube-api-access-zlf8r\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:41.935119 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.934848    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-tmp-dir\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:41.935119 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:41.934857    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99676424-68b7-4256-990e-734e5c3ba73b-dshm\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:42.681075 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:42.681049    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"
Jul 22 19:30:42.681481 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:42.681069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46" event={"ID":"99676424-68b7-4256-990e-734e5c3ba73b","Type":"ContainerDied","Data":"8f55fd28f33823b13093daaf1293566d63f166a5aa316a7c4e655e8707b64b1d"}
Jul 22 19:30:42.681481 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:42.681127    2576 scope.go:117] "RemoveContainer" containerID="d042ffb8cc318e9fb1f9c344820b4e2ac4d45fd9d33a92b57ad5b304cb41b192"
Jul 22 19:30:42.702983 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:42.702956    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"]
Jul 22 19:30:42.706380 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:42.706360    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4ac498/rate-test-model-4ac498-kserve-78754f59f9-4bt46"]
Jul 22 19:30:43.688053 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:43.688022    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99676424-68b7-4256-990e-734e5c3ba73b" path="/var/lib/kubelet/pods/99676424-68b7-4256-990e-734e5c3ba73b/volumes"
Jul 22 19:30:57.792229 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:57.792153    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"]
Jul 22 19:30:57.792674 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:57.792400    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz" podUID="b7b65f11-b82c-40c7-a27b-eb5185523a92" containerName="maas-api" containerID="cri-o://4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6" gracePeriod=30
Jul 22 19:30:57.809653 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:57.809601    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz"]
Jul 22 19:30:57.815636 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:57.815603    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-4ac498-29745810-g65hz"]
Jul 22 19:30:58.043918 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.043854    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:30:58.178447 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.178416    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7b65f11-b82c-40c7-a27b-eb5185523a92-openshift-service-ca\") pod \"b7b65f11-b82c-40c7-a27b-eb5185523a92\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") "
Jul 22 19:30:58.178595 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.178470    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7b65f11-b82c-40c7-a27b-eb5185523a92-maas-api-tls\") pod \"b7b65f11-b82c-40c7-a27b-eb5185523a92\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") "
Jul 22 19:30:58.178595 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.178498    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnw2v\" (UniqueName: \"kubernetes.io/projected/b7b65f11-b82c-40c7-a27b-eb5185523a92-kube-api-access-gnw2v\") pod \"b7b65f11-b82c-40c7-a27b-eb5185523a92\" (UID: \"b7b65f11-b82c-40c7-a27b-eb5185523a92\") "
Jul 22 19:30:58.178851 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.178830    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7b65f11-b82c-40c7-a27b-eb5185523a92-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7b65f11-b82c-40c7-a27b-eb5185523a92" (UID: "b7b65f11-b82c-40c7-a27b-eb5185523a92"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:30:58.180606 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.180576    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7b65f11-b82c-40c7-a27b-eb5185523a92-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7b65f11-b82c-40c7-a27b-eb5185523a92" (UID: "b7b65f11-b82c-40c7-a27b-eb5185523a92"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:30:58.180800 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.180777    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7b65f11-b82c-40c7-a27b-eb5185523a92-kube-api-access-gnw2v" (OuterVolumeSpecName: "kube-api-access-gnw2v") pod "b7b65f11-b82c-40c7-a27b-eb5185523a92" (UID: "b7b65f11-b82c-40c7-a27b-eb5185523a92"). InnerVolumeSpecName "kube-api-access-gnw2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:30:58.280139 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.280110    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7b65f11-b82c-40c7-a27b-eb5185523a92-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:58.280139 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.280136    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7b65f11-b82c-40c7-a27b-eb5185523a92-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:58.280139 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.280145    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnw2v\" (UniqueName: \"kubernetes.io/projected/b7b65f11-b82c-40c7-a27b-eb5185523a92-kube-api-access-gnw2v\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:30:58.762787 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.762751    2576 generic.go:356] "Generic (PLEG): container finished" podID="b7b65f11-b82c-40c7-a27b-eb5185523a92" containerID="4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6" exitCode=0
Jul 22 19:30:58.762962 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.762798    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"
Jul 22 19:30:58.762962 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.762836    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz" event={"ID":"b7b65f11-b82c-40c7-a27b-eb5185523a92","Type":"ContainerDied","Data":"4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6"}
Jul 22 19:30:58.762962 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.762885    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz" event={"ID":"b7b65f11-b82c-40c7-a27b-eb5185523a92","Type":"ContainerDied","Data":"a47e57499073b8f34c6f8883fe614b808c9aff3fe838bad778bf6fea5a15f11e"}
Jul 22 19:30:58.762962 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.762906    2576 scope.go:117] "RemoveContainer" containerID="4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6"
Jul 22 19:30:58.772125 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.772106    2576 scope.go:117] "RemoveContainer" containerID="4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6"
Jul 22 19:30:58.772369 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:30:58.772349    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6\": container with ID starting with 4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6 not found: ID does not exist" containerID="4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6"
Jul 22 19:30:58.772415 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.772378    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6"} err="failed to get container status \"4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6\": rpc error: code = NotFound desc = could not find container \"4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6\": container with ID starting with 4a3cdf2daa84a6a5777079d48935e85d0949370053e996caccb2706202cddcb6 not found: ID does not exist"
Jul 22 19:30:58.784261 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.784234    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"]
Jul 22 19:30:58.786080 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:58.786060    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4ac498-556465d6d-7hsvz"]
Jul 22 19:30:59.688589 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:59.688559    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91c373e9-2a6f-4dca-88ee-f98af78a0338" path="/var/lib/kubelet/pods/91c373e9-2a6f-4dca-88ee-f98af78a0338/volumes"
Jul 22 19:30:59.689066 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:30:59.688938    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7b65f11-b82c-40c7-a27b-eb5185523a92" path="/var/lib/kubelet/pods/b7b65f11-b82c-40c7-a27b-eb5185523a92/volumes"
Jul 22 19:31:20.084224 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.083018    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"]
Jul 22 19:31:20.084224 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.083324    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w" podUID="a3e50e0f-799a-443d-8013-7fe5d729e756" containerName="maas-api" containerID="cri-o://fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a" gracePeriod=30
Jul 22 19:31:20.107896 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.107846    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv"]
Jul 22 19:31:20.111338 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.111311    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8f2692-29745810-9ggfv"]
Jul 22 19:31:20.530350 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.530322    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:31:20.578922 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.578891    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3e50e0f-799a-443d-8013-7fe5d729e756-openshift-service-ca\") pod \"a3e50e0f-799a-443d-8013-7fe5d729e756\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") "
Jul 22 19:31:20.579069 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.578940    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdjfr\" (UniqueName: \"kubernetes.io/projected/a3e50e0f-799a-443d-8013-7fe5d729e756-kube-api-access-hdjfr\") pod \"a3e50e0f-799a-443d-8013-7fe5d729e756\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") "
Jul 22 19:31:20.579069 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.579006    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3e50e0f-799a-443d-8013-7fe5d729e756-maas-api-tls\") pod \"a3e50e0f-799a-443d-8013-7fe5d729e756\" (UID: \"a3e50e0f-799a-443d-8013-7fe5d729e756\") "
Jul 22 19:31:20.579306 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.579283    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3e50e0f-799a-443d-8013-7fe5d729e756-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3e50e0f-799a-443d-8013-7fe5d729e756" (UID: "a3e50e0f-799a-443d-8013-7fe5d729e756"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:31:20.581065 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.581035    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3e50e0f-799a-443d-8013-7fe5d729e756-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3e50e0f-799a-443d-8013-7fe5d729e756" (UID: "a3e50e0f-799a-443d-8013-7fe5d729e756"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:31:20.581182 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.581064    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e50e0f-799a-443d-8013-7fe5d729e756-kube-api-access-hdjfr" (OuterVolumeSpecName: "kube-api-access-hdjfr") pod "a3e50e0f-799a-443d-8013-7fe5d729e756" (UID: "a3e50e0f-799a-443d-8013-7fe5d729e756"). InnerVolumeSpecName "kube-api-access-hdjfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:31:20.680646 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.680599    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3e50e0f-799a-443d-8013-7fe5d729e756-openshift-service-ca\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:31:20.680796 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.680654    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdjfr\" (UniqueName: \"kubernetes.io/projected/a3e50e0f-799a-443d-8013-7fe5d729e756-kube-api-access-hdjfr\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:31:20.680796 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.680667    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3e50e0f-799a-443d-8013-7fe5d729e756-maas-api-tls\") on node \"ip-10-0-139-185.ec2.internal\" DevicePath \"\""
Jul 22 19:31:20.868772 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.868740    2576 generic.go:356] "Generic (PLEG): container finished" podID="a3e50e0f-799a-443d-8013-7fe5d729e756" containerID="fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a" exitCode=0
Jul 22 19:31:20.868942 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.868779    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"
Jul 22 19:31:20.868942 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.868793    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w" event={"ID":"a3e50e0f-799a-443d-8013-7fe5d729e756","Type":"ContainerDied","Data":"fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a"}
Jul 22 19:31:20.868942 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.868837    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w" event={"ID":"a3e50e0f-799a-443d-8013-7fe5d729e756","Type":"ContainerDied","Data":"00bf7605e201e5f1ad1b31db34ac0a3fd1fb49bf68a5231a822ab7461eee2ddb"}
Jul 22 19:31:20.868942 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.868859    2576 scope.go:117] "RemoveContainer" containerID="fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a"
Jul 22 19:31:20.880043 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.880019    2576 scope.go:117] "RemoveContainer" containerID="fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a"
Jul 22 19:31:20.880468 ip-10-0-139-185 kubenswrapper[2576]: E0722 19:31:20.880444    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a\": container with ID starting with fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a not found: ID does not exist" containerID="fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a"
Jul 22 19:31:20.880570 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.880476    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a"} err="failed to get container status \"fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a\": rpc error: code = NotFound desc = could not find container \"fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a\": container with ID starting with fea0cbf53a4aa92e16365b2e6a798d1d035498ddff2ca058e7e2e280a4811f5a not found: ID does not exist"
Jul 22 19:31:20.889543 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.889521    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"]
Jul 22 19:31:20.893490 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:20.893469    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8f2692-bbfb6cf59-nbb8w"]
Jul 22 19:31:21.687724 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:21.687690    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cd98480-269d-4eb9-a96b-80de3aab98b0" path="/var/lib/kubelet/pods/5cd98480-269d-4eb9-a96b-80de3aab98b0/volumes"
Jul 22 19:31:21.688097 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:21.688086    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3e50e0f-799a-443d-8013-7fe5d729e756" path="/var/lib/kubelet/pods/a3e50e0f-799a-443d-8013-7fe5d729e756/volumes"
Jul 22 19:31:31.885910 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:31.885870    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-646cd68c8c-6j6lb_e8a987da-fc08-4c4a-a741-98e4843d5986/authorino/0.log"
Jul 22 19:31:39.064097 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:39.064061    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bmwbj_8a49525f-d272-43bf-adec-c67966c62812/manager/0.log"
Jul 22 19:31:39.215731 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:39.215698    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-9d64f8564-pggwv_630835cd-bd64-4087-9cca-b3d042fb2c1e/manager/0.log"
Jul 22 19:31:39.339606 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:39.339535    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-78rqr_fb0aaeba-b112-47ef-a6ac-3fecfc1277c8/server/0.log"
Jul 22 19:31:39.472107 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:39.472080    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8xrd8_3eb13c22-c477-4dab-99c5-37f8936c122c/manager/0.log"
Jul 22 19:31:39.851830 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:39.851800    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9bb58876-qwvll_6126ec5f-9406-43de-b0d5-8b37a22000c8/copy-manifests/0.log"
Jul 22 19:31:39.870244 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:39.870210    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9bb58876-qwvll_6126ec5f-9406-43de-b0d5-8b37a22000c8/manager/0.log"
Jul 22 19:31:40.257816 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:40.257778    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-75bd6d7f99-msxr2_06a46609-e9b0-4539-b0f2-446b3ddb7fc5/maas-api/0.log"
Jul 22 19:31:40.489956 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:40.489931    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745795-b9ql5_0096aa46-29d2-4595-acbb-95b92c1cf992/cleanup/0.log"
Jul 22 19:31:40.612298 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:40.612211    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745810-xn5bq_1ca8f4f9-e0e7-4fe1-9fd5-394c3a7734a8/cleanup/0.log"
Jul 22 19:31:40.730856 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:40.730825    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5x299_43abc0ed-8a8d-46cd-9c29-1e0dfac0f274/postgres/0.log"
Jul 22 19:31:42.365844 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:42.365809    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-646cd68c8c-6j6lb_e8a987da-fc08-4c4a-a741-98e4843d5986/authorino/0.log"
Jul 22 19:31:42.497814 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:42.497780    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rjl2x_5e9062bc-e607-4431-bda2-cb820b89c98d/manager/0.log"
Jul 22 19:31:42.616524 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:42.616423    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wkjqb_cdb8da78-da94-4dce-b767-d65cd67512ec/manager/0.log"
Jul 22 19:31:42.738590 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:42.738564    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-w7rbs_696d68e9-7f67-4900-afa2-03c26e88863f/kuadrant-console-plugin/0.log"
Jul 22 19:31:42.870107 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:42.870037    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h4qpd_36fac68c-df61-4246-b37c-66c8772335dc/registry-server/0.log"
Jul 22 19:31:42.998864 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:42.998833    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zst2d_9febba1d-e8fe-478c-95fb-28542fac8300/manager/0.log"
Jul 22 19:31:43.620148 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:43.620112    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44c6gs_79c6f2b5-cb56-4b6d-8b3f-be315b55ef73/istio-proxy/0.log"
Jul 22 19:31:44.121461 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:44.121429    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779ddf9494-p2scd_e64c1ba6-a192-4278-bbfd-7cdfa9c64af4/kube-auth-proxy/0.log"
Jul 22 19:31:44.248977 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:44.248943    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jrkgs_f6891e84-95bd-4474-bc17-43fd7b17f2c9/istio-proxy/1.log"
Jul 22 19:31:44.516247 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:44.516159    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-p9785_099ade4d-ef1b-4d80-ab1c-8ba5a61f4946/payload-processing/0.log"
Jul 22 19:31:45.140700 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:45.140669    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5nwlw_6265cb97-fd13-46dd-b488-b25cec49b376/main/0.log"
Jul 22 19:31:45.373252 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:45.373220    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzk2ds_8de2e7b6-fc3f-4717-a685-50fc0e66512a/main/0.log"
Jul 22 19:31:52.284810 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.284780    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4nfc7/must-gather-4z2fd"]
Jul 22 19:31:52.285230 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285124    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7b65f11-b82c-40c7-a27b-eb5185523a92" containerName="maas-api"
Jul 22 19:31:52.285230 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285134    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7b65f11-b82c-40c7-a27b-eb5185523a92" containerName="maas-api"
Jul 22 19:31:52.285230 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285149    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99676424-68b7-4256-990e-734e5c3ba73b" containerName="main"
Jul 22 19:31:52.285230 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285154    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="99676424-68b7-4256-990e-734e5c3ba73b" containerName="main"
Jul 22 19:31:52.285230 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285161    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83a8e0cf-64b1-443d-a6d0-666aa1bfff11" containerName="test-curl-7789-140041791286960"
Jul 22 19:31:52.285230 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285168    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a8e0cf-64b1-443d-a6d0-666aa1bfff11" containerName="test-curl-7789-140041791286960"
Jul 22 19:31:52.285230 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285178    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e50e0f-799a-443d-8013-7fe5d729e756" containerName="maas-api"
Jul 22 19:31:52.285230 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285185    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e50e0f-799a-443d-8013-7fe5d729e756" containerName="maas-api"
Jul 22 19:31:52.285611 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285263    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e50e0f-799a-443d-8013-7fe5d729e756" containerName="maas-api"
Jul 22 19:31:52.285611 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285278    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="83a8e0cf-64b1-443d-a6d0-666aa1bfff11" containerName="test-curl-7789-140041791286960"
Jul 22 19:31:52.285611 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285287    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="99676424-68b7-4256-990e-734e5c3ba73b" containerName="main"
Jul 22 19:31:52.285611 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.285297    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7b65f11-b82c-40c7-a27b-eb5185523a92" containerName="maas-api"
Jul 22 19:31:52.288622 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.288605    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nfc7/must-gather-4z2fd"
Jul 22 19:31:52.291463 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.291440    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4nfc7\"/\"kube-root-ca.crt\""
Jul 22 19:31:52.291643 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.291609    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4nfc7\"/\"openshift-service-ca.crt\""
Jul 22 19:31:52.292818 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.292802    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4nfc7\"/\"default-dockercfg-dxrwg\""
Jul 22 19:31:52.296286 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.296266    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nfc7/must-gather-4z2fd"]
Jul 22 19:31:52.454682 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.454624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a3a3059-b061-40e6-9820-3d99ba8c7c48-must-gather-output\") pod \"must-gather-4z2fd\" (UID: \"5a3a3059-b061-40e6-9820-3d99ba8c7c48\") " pod="openshift-must-gather-4nfc7/must-gather-4z2fd"
Jul 22 19:31:52.454852 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.454719    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lmjj\" (UniqueName: \"kubernetes.io/projected/5a3a3059-b061-40e6-9820-3d99ba8c7c48-kube-api-access-2lmjj\") pod \"must-gather-4z2fd\" (UID: \"5a3a3059-b061-40e6-9820-3d99ba8c7c48\") " pod="openshift-must-gather-4nfc7/must-gather-4z2fd"
Jul 22 19:31:52.556061 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.555977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lmjj\" (UniqueName: \"kubernetes.io/projected/5a3a3059-b061-40e6-9820-3d99ba8c7c48-kube-api-access-2lmjj\") pod \"must-gather-4z2fd\" (UID: \"5a3a3059-b061-40e6-9820-3d99ba8c7c48\") " pod="openshift-must-gather-4nfc7/must-gather-4z2fd"
Jul 22 19:31:52.556202 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.556066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a3a3059-b061-40e6-9820-3d99ba8c7c48-must-gather-output\") pod \"must-gather-4z2fd\" (UID: \"5a3a3059-b061-40e6-9820-3d99ba8c7c48\") " pod="openshift-must-gather-4nfc7/must-gather-4z2fd"
Jul 22 19:31:52.556401 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.556382    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a3a3059-b061-40e6-9820-3d99ba8c7c48-must-gather-output\") pod \"must-gather-4z2fd\" (UID: \"5a3a3059-b061-40e6-9820-3d99ba8c7c48\") " pod="openshift-must-gather-4nfc7/must-gather-4z2fd"
Jul 22 19:31:52.564107 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.564083    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lmjj\" (UniqueName: \"kubernetes.io/projected/5a3a3059-b061-40e6-9820-3d99ba8c7c48-kube-api-access-2lmjj\") pod \"must-gather-4z2fd\" (UID: \"5a3a3059-b061-40e6-9820-3d99ba8c7c48\") " pod="openshift-must-gather-4nfc7/must-gather-4z2fd"
Jul 22 19:31:52.598785 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.598760    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nfc7/must-gather-4z2fd"
Jul 22 19:31:52.846993 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:52.846916    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nfc7/must-gather-4z2fd"]
Jul 22 19:31:52.872340 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:31:52.872313    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a3a3059_b061_40e6_9820_3d99ba8c7c48.slice/crio-a0d01a1a4f498aa575b0ea77b1512d8a3b154a7721627a45b19c35596aa2ae46 WatchSource:0}: Error finding container a0d01a1a4f498aa575b0ea77b1512d8a3b154a7721627a45b19c35596aa2ae46: Status 404 returned error can't find the container with id a0d01a1a4f498aa575b0ea77b1512d8a3b154a7721627a45b19c35596aa2ae46
Jul 22 19:31:53.029076 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:53.029041    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nfc7/must-gather-4z2fd" event={"ID":"5a3a3059-b061-40e6-9820-3d99ba8c7c48","Type":"ContainerStarted","Data":"a0d01a1a4f498aa575b0ea77b1512d8a3b154a7721627a45b19c35596aa2ae46"}
Jul 22 19:31:54.038044 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:54.037943    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nfc7/must-gather-4z2fd" event={"ID":"5a3a3059-b061-40e6-9820-3d99ba8c7c48","Type":"ContainerStarted","Data":"c855171489ec92cdd63c17a63fc0c9792f9c4c0b96643afd6001178a0caa3bd6"}
Jul 22 19:31:54.038044 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:54.037998    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nfc7/must-gather-4z2fd" event={"ID":"5a3a3059-b061-40e6-9820-3d99ba8c7c48","Type":"ContainerStarted","Data":"fb88f2c2c70b965a1816c4df20aa373997488d9cff60b0802db9eea24e6bb10e"}
Jul 22 19:31:54.055478 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:54.055416    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4nfc7/must-gather-4z2fd" podStartSLOduration=1.170427404 podStartE2EDuration="2.055395985s" totalImagesPullingTime="884.968581ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:31:54.053249468 +0000 UTC m=+3114.943790400" watchObservedRunningTime="2026-07-22 19:31:54.055395985 +0000 UTC m=+3114.945936918"
Jul 22 19:31:55.336139 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:55.336106    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-shh65_04e18ad9-ab77-4ea3-b3b7-3c2a570e096b/global-pull-secret-syncer/0.log"
Jul 22 19:31:55.468264 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:55.468234    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nxss6_3a6d9e14-95eb-4686-ab67-5ca7aff3011e/konnectivity-agent/0.log"
Jul 22 19:31:55.514642 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:31:55.514589    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-185.ec2.internal_56d73fc2a3618a7351db3712287acb62/haproxy/0.log"
Jul 22 19:32:00.442759 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:00.442720    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-646cd68c8c-6j6lb_e8a987da-fc08-4c4a-a741-98e4843d5986/authorino/0.log"
Jul 22 19:32:00.475612 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:00.475578    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rjl2x_5e9062bc-e607-4431-bda2-cb820b89c98d/manager/0.log"
Jul 22 19:32:00.511962 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:00.511931    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wkjqb_cdb8da78-da94-4dce-b767-d65cd67512ec/manager/0.log"
Jul 22 19:32:00.545126 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:00.545097    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-w7rbs_696d68e9-7f67-4900-afa2-03c26e88863f/kuadrant-console-plugin/0.log"
Jul 22 19:32:00.601203 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:00.601166    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h4qpd_36fac68c-df61-4246-b37c-66c8772335dc/registry-server/0.log"
Jul 22 19:32:00.828699 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:00.828019    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zst2d_9febba1d-e8fe-478c-95fb-28542fac8300/manager/0.log"
Jul 22 19:32:02.810648 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:02.810251    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vlm28_6ba43672-693f-40b1-831a-545fc97ad5f4/node-exporter/0.log"
Jul 22 19:32:02.837585 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:02.837552    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vlm28_6ba43672-693f-40b1-831a-545fc97ad5f4/kube-rbac-proxy/0.log"
Jul 22 19:32:02.861254 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:02.861221    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vlm28_6ba43672-693f-40b1-831a-545fc97ad5f4/init-textfile/0.log"
Jul 22 19:32:02.890238 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:02.890210    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-htcvp_3fe71b81-c0de-4ea0-92a5-332e60d78b99/kube-rbac-proxy-main/0.log"
Jul 22 19:32:02.918073 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:02.918044    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-htcvp_3fe71b81-c0de-4ea0-92a5-332e60d78b99/kube-rbac-proxy-self/0.log"
Jul 22 19:32:02.943258 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:02.943224    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-htcvp_3fe71b81-c0de-4ea0-92a5-332e60d78b99/openshift-state-metrics/0.log"
Jul 22 19:32:06.338118 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.338077    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"]
Jul 22 19:32:06.344946 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.344913    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.346734 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.346707    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"]
Jul 22 19:32:06.395788 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.395720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnl56\" (UniqueName: \"kubernetes.io/projected/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-kube-api-access-fnl56\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.395990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.395821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-podres\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.395990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.395851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-lib-modules\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.395990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.395887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-proc\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.395990 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.395935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-sys\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.497030 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.496993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-sys\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.497219 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.497102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnl56\" (UniqueName: \"kubernetes.io/projected/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-kube-api-access-fnl56\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.497219 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.497151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-sys\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.497219 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.497180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-podres\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.497219 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.497208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-lib-modules\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.497467 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.497250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-proc\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.497467 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.497344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-proc\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.497467 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.497463    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-podres\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.497660 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.497553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-lib-modules\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.506141 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.506117    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnl56\" (UniqueName: \"kubernetes.io/projected/ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a-kube-api-access-fnl56\") pod \"perf-node-gather-daemonset-59ldv\" (UID: \"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.658164 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.658130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:06.845972 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:06.844063    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"]
Jul 22 19:32:06.876234 ip-10-0-139-185 kubenswrapper[2576]: W0722 19:32:06.876120    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podad8421b2_8eb6_4c9e_aac3_d0f4f30fb19a.slice/crio-94eae7e1774d9e630e45cce1d0dab0c10418689b3e412af9d0a775b500649883 WatchSource:0}: Error finding container 94eae7e1774d9e630e45cce1d0dab0c10418689b3e412af9d0a775b500649883: Status 404 returned error can't find the container with id 94eae7e1774d9e630e45cce1d0dab0c10418689b3e412af9d0a775b500649883
Jul 22 19:32:07.142161 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:07.142126    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:07.142389 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:07.142363    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv" event={"ID":"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a","Type":"ContainerStarted","Data":"9d3fd716e7dfab182b09660d157b51309c953553a113064dc83f52c8267bc7e3"}
Jul 22 19:32:07.142464 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:07.142410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv" event={"ID":"ad8421b2-8eb6-4c9e-aac3-d0f4f30fb19a","Type":"ContainerStarted","Data":"94eae7e1774d9e630e45cce1d0dab0c10418689b3e412af9d0a775b500649883"}
Jul 22 19:32:07.158413 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:07.158348    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv" podStartSLOduration=1.158327495 podStartE2EDuration="1.158327495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 19:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:32:07.157564599 +0000 UTC m=+3128.048105532" watchObservedRunningTime="2026-07-22 19:32:07.158327495 +0000 UTC m=+3128.048868429"
Jul 22 19:32:07.680192 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:07.680162    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vpz2f_2986a3e4-da73-4650-9389-a4d1a1f22302/dns/0.log"
Jul 22 19:32:07.699056 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:07.699024    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vpz2f_2986a3e4-da73-4650-9389-a4d1a1f22302/kube-rbac-proxy/0.log"
Jul 22 19:32:07.763122 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:07.763091    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4kzc7_ba8ebbb4-4eac-46c0-a872-5b3ea8d447ab/dns-node-resolver/0.log"
Jul 22 19:32:08.321780 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:08.321752    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-65d7fc9fc6-j56rc_8be5a1ca-3804-49cc-8ce9-5daae8826765/registry/0.log"
Jul 22 19:32:08.345825 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:08.345797    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4p574_89019e31-6d29-49fd-8fc3-f486fd2c1209/node-ca/0.log"
Jul 22 19:32:09.280342 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:09.280306    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44c6gs_79c6f2b5-cb56-4b6d-8b3f-be315b55ef73/istio-proxy/0.log"
Jul 22 19:32:09.452766 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:09.452738    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779ddf9494-p2scd_e64c1ba6-a192-4278-bbfd-7cdfa9c64af4/kube-auth-proxy/0.log"
Jul 22 19:32:09.480368 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:09.480339    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jrkgs_f6891e84-95bd-4474-bc17-43fd7b17f2c9/istio-proxy/0.log"
Jul 22 19:32:09.537286 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:09.537212    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jrkgs_f6891e84-95bd-4474-bc17-43fd7b17f2c9/istio-proxy/1.log"
Jul 22 19:32:09.617807 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:09.617777    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-p9785_099ade4d-ef1b-4d80-ab1c-8ba5a61f4946/payload-processing/0.log"
Jul 22 19:32:10.329102 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:10.329066    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9h9qz_5241d666-4a13-4c9d-8ff2-a7448dce1332/serve-healthcheck-canary/0.log"
Jul 22 19:32:10.810858 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:10.810822    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cg27c_70d227ba-ecb7-446b-a997-e770e52d894d/kube-rbac-proxy/0.log"
Jul 22 19:32:10.830686 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:10.830657    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cg27c_70d227ba-ecb7-446b-a997-e770e52d894d/exporter/0.log"
Jul 22 19:32:10.850108 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:10.850080    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cg27c_70d227ba-ecb7-446b-a997-e770e52d894d/extractor/0.log"
Jul 22 19:32:13.328026 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:13.327996    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bmwbj_8a49525f-d272-43bf-adec-c67966c62812/manager/0.log"
Jul 22 19:32:13.547792 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:13.547753    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-9d64f8564-pggwv_630835cd-bd64-4087-9cca-b3d042fb2c1e/manager/0.log"
Jul 22 19:32:13.567757 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:13.567730    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-78rqr_fb0aaeba-b112-47ef-a6ac-3fecfc1277c8/server/0.log"
Jul 22 19:32:13.686220 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:13.686150    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8xrd8_3eb13c22-c477-4dab-99c5-37f8936c122c/manager/0.log"
Jul 22 19:32:13.922803 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:13.922770    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9bb58876-qwvll_6126ec5f-9406-43de-b0d5-8b37a22000c8/manager/0.log"
Jul 22 19:32:13.937497 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:13.937428    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9bb58876-qwvll_6126ec5f-9406-43de-b0d5-8b37a22000c8/copy-manifests/0.log"
Jul 22 19:32:14.166748 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:14.166722    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-59ldv"
Jul 22 19:32:15.234079 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:15.234038    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-fhsx8_43ec3b4f-8632-4955-ba17-3765d176bc16/openshift-lws-operator/0.log"
Jul 22 19:32:15.553549 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:15.553461    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44c6gs_79c6f2b5-cb56-4b6d-8b3f-be315b55ef73/istio-proxy/0.log"
Jul 22 19:32:15.711790 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:15.711761    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779ddf9494-p2scd_e64c1ba6-a192-4278-bbfd-7cdfa9c64af4/kube-auth-proxy/0.log"
Jul 22 19:32:15.733421 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:15.733389    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jrkgs_f6891e84-95bd-4474-bc17-43fd7b17f2c9/istio-proxy/0.log"
Jul 22 19:32:15.787615 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:15.787575    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jrkgs_f6891e84-95bd-4474-bc17-43fd7b17f2c9/istio-proxy/1.log"
Jul 22 19:32:15.851073 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:15.851038    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-p9785_099ade4d-ef1b-4d80-ab1c-8ba5a61f4946/payload-processing/0.log"
Jul 22 19:32:17.033472 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:17.033440    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44c6gs_79c6f2b5-cb56-4b6d-8b3f-be315b55ef73/istio-proxy/0.log"
Jul 22 19:32:17.184489 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:17.184454    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779ddf9494-p2scd_e64c1ba6-a192-4278-bbfd-7cdfa9c64af4/kube-auth-proxy/0.log"
Jul 22 19:32:17.209166 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:17.209137    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jrkgs_f6891e84-95bd-4474-bc17-43fd7b17f2c9/istio-proxy/0.log"
Jul 22 19:32:17.256239 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:17.255614    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jrkgs_f6891e84-95bd-4474-bc17-43fd7b17f2c9/istio-proxy/1.log"
Jul 22 19:32:17.336947 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:17.336919    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-p9785_099ade4d-ef1b-4d80-ab1c-8ba5a61f4946/payload-processing/0.log"
Jul 22 19:32:21.737562 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:21.737536    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qzlbl_f88707f9-0289-4032-a079-321f82e7c4e0/kube-multus-additional-cni-plugins/0.log"
Jul 22 19:32:21.756021 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:21.755995    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qzlbl_f88707f9-0289-4032-a079-321f82e7c4e0/egress-router-binary-copy/0.log"
Jul 22 19:32:21.774161 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:21.774141    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qzlbl_f88707f9-0289-4032-a079-321f82e7c4e0/cni-plugins/0.log"
Jul 22 19:32:21.793027 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:21.793005    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qzlbl_f88707f9-0289-4032-a079-321f82e7c4e0/bond-cni-plugin/0.log"
Jul 22 19:32:21.818175 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:21.818158    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qzlbl_f88707f9-0289-4032-a079-321f82e7c4e0/routeoverride-cni/0.log"
Jul 22 19:32:21.837330 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:21.837311    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qzlbl_f88707f9-0289-4032-a079-321f82e7c4e0/whereabouts-cni-bincopy/0.log"
Jul 22 19:32:21.856276 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:21.856249    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qzlbl_f88707f9-0289-4032-a079-321f82e7c4e0/whereabouts-cni/0.log"
Jul 22 19:32:22.075089 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:22.074947    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wgr2l_412b6a94-062d-4f4f-a8b7-0c031cb16908/kube-multus/0.log"
Jul 22 19:32:22.210756 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:22.210720    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xd9sx_881e051a-b4aa-4d47-b389-0f3dd300de2f/network-metrics-daemon/0.log"
Jul 22 19:32:22.225505 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:22.225481    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xd9sx_881e051a-b4aa-4d47-b389-0f3dd300de2f/kube-rbac-proxy/0.log"
Jul 22 19:32:23.439538 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:23.439501    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c88z2_bd157c08-d963-4c18-a43d-16418f13baf4/ovn-controller/0.log"
Jul 22 19:32:23.456684 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:23.456658    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c88z2_bd157c08-d963-4c18-a43d-16418f13baf4/ovn-acl-logging/0.log"
Jul 22 19:32:23.488022 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:23.487991    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c88z2_bd157c08-d963-4c18-a43d-16418f13baf4/kube-rbac-proxy-node/0.log"
Jul 22 19:32:23.511515 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:23.511491    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c88z2_bd157c08-d963-4c18-a43d-16418f13baf4/kube-rbac-proxy-ovn-metrics/0.log"
Jul 22 19:32:23.528458 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:23.528437    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c88z2_bd157c08-d963-4c18-a43d-16418f13baf4/northd/0.log"
Jul 22 19:32:23.553025 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:23.552996    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c88z2_bd157c08-d963-4c18-a43d-16418f13baf4/nbdb/0.log"
Jul 22 19:32:23.577428 ip-10-0-139-185 kubenswrapper[2576]: I0722 19:32:23.577404    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c88z2_bd157c08-d963-4c18-a43d-16418f13baf4/sbdb/0.log"