Jul 23 12:08:04.737131 ip-10-0-137-16 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 23 12:08:04.737143 ip-10-0-137-16 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 23 12:08:04.737152 ip-10-0-137-16 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 23 12:08:04.737499 ip-10-0-137-16 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 23 12:08:15.822682 ip-10-0-137-16 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 23 12:08:15.822698 ip-10-0-137-16 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 19203dc594c040b9a8d76efd0dba4d00 --
Jul 23 12:10:16.186028 ip-10-0-137-16 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 12:10:16.754066 ip-10-0-137-16 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 12:10:16.754066 ip-10-0-137-16 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 12:10:16.754066 ip-10-0-137-16 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 12:10:16.754066 ip-10-0-137-16 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 12:10:16.754066 ip-10-0-137-16 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 12:10:16.755241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.755147    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 12:10:16.760191 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760173    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 12:10:16.760191 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760191    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760195    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760199    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760203    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760206    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760209    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760212    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760215    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760218    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760221    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760224    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760227    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760229    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760232    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760235    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760238    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760241    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760243    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760246    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760249    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 12:10:16.760264 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760252    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760255    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760258    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760260    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760266    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760270    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760273    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760276    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760279    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760282    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760285    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760287    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760290    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760292    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760295    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760298    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760301    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760304    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760307    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 12:10:16.760779 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760311    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760316    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760319    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760322    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760325    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760328    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760330    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760333    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760337    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760340    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760342    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760345    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760348    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760351    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760355    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760359    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760361    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760364    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760368    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760371    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 12:10:16.761271 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760374    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760377    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760380    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760384    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760386    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760389    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760392    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760394    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760397    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760400    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760403    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760405    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760408    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760411    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760413    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760416    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760419    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760421    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760424    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760427    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 12:10:16.761813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760429    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760432    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760435    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760437    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760440    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760443    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760446    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760450    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760453    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760458    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760462    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760465    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760468    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760471    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760475    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760479    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760482    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760485    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.760488    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 12:10:16.762361 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761270    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761279    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761283    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761287    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761290    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761293    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761298    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761302    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761306    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761309    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761312    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761315    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761318    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761321    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761324    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761327    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761329    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761332    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761335    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761338    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 12:10:16.762925 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761342    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761345    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761348    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761351    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761354    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761357    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761360    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761363    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761366    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761369    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761372    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761374    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761377    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761380    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761382    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761385    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761389    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761393    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761395    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761398    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 12:10:16.763459 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761401    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761403    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761406    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761410    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761412    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761415    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761418    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761421    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761423    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761426    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761429    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761434    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761438    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761441    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761444    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761446    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761449    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761452    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761455    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761458    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 12:10:16.764019 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761460    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761463    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761466    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761468    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761471    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761474    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761476    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761479    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761482    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761485    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761487    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761490    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761493    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761495    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761498    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761502    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761505    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761507    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761511    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 12:10:16.764546 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761513    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761516    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761519    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761521    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761524    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761527    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761530    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761533    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761536    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761539    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761542    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761545    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761548    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761554    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761558    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761561    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761564    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761567    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761570    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.761572    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 12:10:16.765062 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762030    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762036    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762040    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762043    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762047    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762050    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762053    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762056    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762060    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762064    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762066    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762069    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762072    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762075    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762085    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762090    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762093    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762096    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762100    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762105    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 12:10:16.765594 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762108    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762112    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762114    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762117    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762121    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762124    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762127    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762129    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762132    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762135    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762137    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762140    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762143    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762146    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762149    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762151    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762154    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762157    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762160    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762163    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 12:10:16.766129 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762166    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762169    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762171    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762174    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762177    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762179    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762182    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762185    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762188    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762190    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762193    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762196    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762199    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762202    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762204    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762209    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762213    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762216    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762219    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 12:10:16.766657 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762222    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762224    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762227    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762230    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762232    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762235    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762238    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762241    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762243    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762246    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762249    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762251    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762254    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762257    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762259    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762262    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762265    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762268    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762270    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762273    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 12:10:16.767161 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762276    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762281    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762284    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762287    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762289    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762292    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762295    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762299    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762302    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762306    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762309    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762313    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762315    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762318    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762321    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762324    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762326    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762329    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762332    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.762335    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 12:10:16.767670 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762404    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762411    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762418    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762423    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762427    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762431    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762436    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762441    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762444    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762448    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762452    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762455    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762458    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762464    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762467    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762470    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762474    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762477    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762481    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762484    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762488    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762491    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762495    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762499    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 12:10:16.768200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762503    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762506    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762510    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762513    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762517    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762520    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762523    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762526    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762531    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762534    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762537    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762540    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762543    2565 flags.go:64] FLAG: --enable-server="true"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762548    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762553    2565 flags.go:64] FLAG: --event-burst="100"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762556    2565 flags.go:64] FLAG: --event-qps="50"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762559    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762563    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762566    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762570    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762574    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762578    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762581    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762584    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762588    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 12:10:16.768814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762591    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762594    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762597    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762600    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762603    2565 flags.go:64] FLAG: --feature-gates=""
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762607    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762610    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762614    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762622    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762626    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762629    2565 flags.go:64] FLAG: --help="false"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762633    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-137-16.ec2.internal"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762636    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762639    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762642    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762646    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762650    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762652    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762655    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762658    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762663    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762666    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762670    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762673    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 23 12:10:16.769526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762676    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762679    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762683    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762686    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762690    2565 flags.go:64] FLAG: --lock-file=""
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762693    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762697    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762700    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762713    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762716    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762719    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762722    2565 flags.go:64] FLAG: --logging-format="text"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762725    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762729    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762732    2565 flags.go:64] FLAG: --manifest-url=""
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762735    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762739    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762743    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762747    2565 flags.go:64] FLAG: --max-pods="110"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762751    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762754    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762757    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762760    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762763    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762766    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 12:10:16.770402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762770    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762777    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762780    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762785    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762788    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762791    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762797    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762800    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762803    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762807    2565 flags.go:64] FLAG: --port="10250"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762810    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762813    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d4968072e7f91d69"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762818    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762821    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762824    2565 flags.go:64] FLAG: --register-node="true"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762828    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762845    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762850    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762855    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762860    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762866    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762871    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762874    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762877    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762880    2565 flags.go:64] FLAG: --runonce="false"
Jul 23 12:10:16.771343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762884    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762888    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762891    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762894    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762897    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762901    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762904    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762907    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762910    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762913    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762918    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762921    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762924    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762927    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762930    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762936    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762940    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762943    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762948    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762952    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762955    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762958    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762961    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762964    2565 flags.go:64] FLAG: --v="2"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762968    2565 flags.go:64] FLAG: --version="false"
Jul 23 12:10:16.772026 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762972    2565 flags.go:64] FLAG: --vmodule=""
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762976    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.762980    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763101    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763105    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763108    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763111    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763116    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763120    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763123    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763126    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763129    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763132    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763135    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763137    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763140    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763143    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763147    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763150    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763153    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 12:10:16.772674 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763156    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763158    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763161    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763164    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763166    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763169    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763173    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763176    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763179    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763181    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763184    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763187    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763190    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763193    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763195    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763198    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763201    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763204    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763207    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 12:10:16.773364 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763209    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763212    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763215    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763218    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763220    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763223    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763226    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763228    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763231    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763234    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763238    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763241    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763244    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763246    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763249    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763252    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763255    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763258    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763260    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763263    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 12:10:16.773922 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763266    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763270    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763272    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763275    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763278    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763280    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763283    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763286    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763288    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763291    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763294    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763296    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763299    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763302    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763304    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763308    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763311    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763313    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763316    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763319    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 12:10:16.774565 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763321    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763325    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763330    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763333    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763336    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763339    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763341    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763344    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763347    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763350    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763354    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763358    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763361    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763365    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763368    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763371    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763373    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763376    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763379    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 12:10:16.775096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763381    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763384    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763387    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.763390    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.764106    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.774475    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.774498    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.774520    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.774529    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774646    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774653    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774660    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774663    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774666    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774670    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 12:10:16.775595 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774673    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774676    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774679    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774682    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774687    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774691    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774695    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774698    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774705    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774709    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774712    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774715    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774718    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774722    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774726    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774730    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774733    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774737    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774740    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 12:10:16.776027 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774743    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774748    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774753    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774757    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774767    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774770    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774774    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774777    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774780    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774783    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774785    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774788    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774791    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774794    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774800    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774803    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774806    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774809    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774812    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774815    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 12:10:16.776511 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774818    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774989    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774995    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.774998    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775001    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775004    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775007    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775009    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775012    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775015    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775018    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775021    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775023    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775026    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775029    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775031    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775035    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775041    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775044    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775047    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 12:10:16.777031 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775050    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775053    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775056    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775059    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775062    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775065    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775067    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775070    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775073    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775076    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775087    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775090    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775093    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775096    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775099    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775102    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775105    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775108    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775111    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775114    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 12:10:16.777556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775116    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775119    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775122    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775125    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775127    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775130    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775133    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775136    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775141    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775144    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775146    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775149    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775152    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775155    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.775161    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 12:10:16.778085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775300    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775305    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775309    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775312    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775315    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775318    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775321    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775324    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775326    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775329    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775332    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775334    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775337    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775340    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775342    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775345    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775348    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775351    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775354    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775357    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 12:10:16.778476 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775360    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775362    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775365    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775368    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775372    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775374    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775378    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775381    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775385    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775389    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775392    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775395    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775399    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775402    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775405    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775408    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775411    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775414    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775416    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775419    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 12:10:16.779011 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775422    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775425    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775428    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775431    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775434    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775444    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775446    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775449    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775452    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775455    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775458    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775460    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775463    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775466    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775469    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775471    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775474    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775477    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775480    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775483    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 12:10:16.779556 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775486    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775489    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775491    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775494    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775496    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775500    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775504    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775507    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775509    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775512    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775515    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775517    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775520    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775522    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775525    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775528    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775530    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775533    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775544    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 12:10:16.780083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775547    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775550    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775552    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775555    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775557    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775560    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775564    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775568    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775571    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775574    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775577    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775580    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775583    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775586    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775588    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775591    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775594    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775597    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775599    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 12:10:16.780567 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:16.775602    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 12:10:16.781068 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.775608    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 12:10:16.781068 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.775724    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 12:10:16.781068 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.777863    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 12:10:16.781068 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.779601    2565 server.go:1019] "Starting client certificate rotation"
Jul 23 12:10:16.781068 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.779714    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 12:10:16.781068 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.779747    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 12:10:16.782942 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.782927    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 12:10:16.784281 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.784255    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 12:10:16.795717 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.795692    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 23 12:10:16.800120 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.800104    2565 log.go:25] "Validated CRI v1 image API"
Jul 23 12:10:16.801872 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.801858    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 12:10:16.804921 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.804900    2565 fs.go:135] Filesystem UUIDs: map[4844d932-8cc5-4f09-9482-dcdc6e56cb46:/dev/nvme0n1p3 6d2f0ede-2330-4467-9645-2afc027ce3bb:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 23 12:10:16.805015 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.804921    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 12:10:16.812470 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.812332    2565 manager.go:216] Machine: {Timestamp:2026-07-23 12:10:16.8094204 +0000 UTC m=+0.464929197 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099868 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ed53723def4f23783a6068d4acffa SystemUUID:ec2ed537-23de-f4f2-3783-a6068d4acffa BootID:19203dc5-94c0-40b9-a8d7-6efd0dba4d00 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9e:80:00:79:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9e:80:00:79:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:d1:41:5d:5a:6b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 12:10:16.812470 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.812456    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 12:10:16.812630 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.812572    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 12:10:16.816214 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.816180    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 12:10:16.816378 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.816216    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-16.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 12:10:16.816466 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.816450    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 12:10:16.818183 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.818171    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 12:10:16.818222 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.818186    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 12:10:16.818222 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.818194    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 12:10:16.819356 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.819342    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 12:10:16.820925 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.820914    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 12:10:16.824504 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.824491    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 12:10:16.824641 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.824631    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 12:10:16.824673 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.824665    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 12:10:16.824708 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.824680    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 12:10:16.824708 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.824693    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 12:10:16.824708 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.824707    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 23 12:10:16.824799 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.824718    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 12:10:16.826284 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.826248    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 12:10:16.826373 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.826299    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 12:10:16.835003 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.834976    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 12:10:16.837560 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.837545    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 12:10:16.837612 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.837578    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 12:10:16.839077 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839064    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 12:10:16.839119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839082    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 12:10:16.839119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839089    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 12:10:16.839119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839095    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 12:10:16.839119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839104    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 12:10:16.839119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839110    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 12:10:16.839119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839118    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 12:10:16.839298 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839123    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 12:10:16.839298 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839131    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 12:10:16.839298 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839137    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 12:10:16.839298 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839146    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 12:10:16.839298 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.839151    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 12:10:16.840702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.840680    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 12:10:16.840702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.840704    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 12:10:16.842238 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.842205    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 12:10:16.842238 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.842209    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 12:10:16.844419 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.844398    2565 server.go:1268] "Started kubelet"
Jul 23 12:10:16.844518 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.844484    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 12:10:16.844593 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.844511    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 12:10:16.844643 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.844589    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 12:10:16.845558 ip-10-0-137-16 systemd[1]: Started Kubernetes Kubelet.
Jul 23 12:10:16.846007 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.845987    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 12:10:16.847052 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.847035    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 12:10:16.850087 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.850060    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 12:10:16.851290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.851273    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2q46l"
Jul 23 12:10:16.852120 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.852103    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-16.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 12:10:16.853433 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.852136    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-16.ec2.internal.18c4e9fb0f109d67  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-16.ec2.internal,UID:ip-10-0-137-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-16.ec2.internal,},FirstTimestamp:2026-07-23 12:10:16.844369255 +0000 UTC m=+0.499878053,LastTimestamp:2026-07-23 12:10:16.844369255 +0000 UTC m=+0.499878053,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-16.ec2.internal,}"
Jul 23 12:10:16.854199 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.854182    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 12:10:16.854199 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.854192    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 12:10:16.854316 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.854260    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:16.854819 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.854783    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 12:10:16.854927 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.854790    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 12:10:16.855027 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855008    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 12:10:16.855149 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855132    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 12:10:16.855149 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855142    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 12:10:16.855240 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855149    2565 factory.go:55] Registering systemd factory
Jul 23 12:10:16.855240 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855165    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 23 12:10:16.855438 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855422    2565 factory.go:153] Registering CRI-O factory
Jul 23 12:10:16.855480 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855443    2565 factory.go:223] Registration of the crio container factory successfully
Jul 23 12:10:16.855517 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855500    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 12:10:16.855552 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855523    2565 factory.go:103] Registering Raw factory
Jul 23 12:10:16.855552 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.855537    2565 manager.go:1196] Started watching for new ooms in manager
Jul 23 12:10:16.856032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.856011    2565 manager.go:349] Starting recovery of all containers
Jul 23 12:10:16.861510 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.861475    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 12:10:16.861654 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.861625    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2q46l"
Jul 23 12:10:16.862333 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.862299    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 12:10:16.869984 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.869845    2565 manager.go:354] Recovery completed
Jul 23 12:10:16.874596 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.874580    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 12:10:16.876665 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.876646    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 12:10:16.876750 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.876676    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 12:10:16.876750 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.876687    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientPID"
Jul 23 12:10:16.877280 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.877265    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 12:10:16.877280 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.877275    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 12:10:16.877366 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.877279    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 12:10:16.877366 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.877318    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 12:10:16.878120 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.878101    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 12:10:16.878193 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.878131    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 12:10:16.878193 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.878141    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientPID"
Jul 23 12:10:16.878730 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.878714    2565 policy_none.go:49] "None policy: Start"
Jul 23 12:10:16.878810 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.878735    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 12:10:16.878810 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.878750    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 12:10:16.878939 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.878779    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-16.ec2.internal.18c4e9fb10fd5d70  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-16.ec2.internal,UID:ip-10-0-137-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-16.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-16.ec2.internal,},FirstTimestamp:2026-07-23 12:10:16.876662128 +0000 UTC m=+0.532170932,LastTimestamp:2026-07-23 12:10:16.876662128 +0000 UTC m=+0.532170932,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-16.ec2.internal,}"
Jul 23 12:10:16.879976 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.879958    2565 policy_none.go:47] "Start"
Jul 23 12:10:16.919449 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.919430    2565 manager.go:337] "Starting Device Plugin manager"
Jul 23 12:10:16.919557 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.919475    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 12:10:16.919557 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.919488    2565 server.go:87] "Starting device plugin registration server"
Jul 23 12:10:16.919800 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.919783    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 12:10:16.919869 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.919818    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 12:10:16.919929 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.919893    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 12:10:16.920000 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.919987    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 12:10:16.920048 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:16.919999    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 12:10:16.920606 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.920580    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 12:10:16.920727 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:16.920643    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.020537 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.020451    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 12:10:17.022151 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.021424    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 12:10:17.022151 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.021452    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 12:10:17.022151 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.021465    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientPID"
Jul 23 12:10:17.022151 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.021493    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.038973 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.038950    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.039048 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.038977    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-16.ec2.internal\": node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.041944 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.041918    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 12:10:17.043301 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.043274    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 12:10:17.043420 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.043316    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 12:10:17.043420 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.043340    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 12:10:17.043420 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.043379    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 12:10:17.049665 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.049648    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 12:10:17.116232 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.116196    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.144291 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.144254    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal"]
Jul 23 12:10:17.144375 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.144338    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 12:10:17.145372 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.145355    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 12:10:17.145478 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.145386    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 12:10:17.145478 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.145397    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientPID"
Jul 23 12:10:17.146886 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.146872    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 12:10:17.146999 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.146983    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.147048 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.147014    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 12:10:17.148224 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.148206    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 12:10:17.148342 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.148238    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 12:10:17.148342 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.148209    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 12:10:17.148342 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.148255    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientPID"
Jul 23 12:10:17.148342 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.148272    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 12:10:17.148342 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.148285    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientPID"
Jul 23 12:10:17.149654 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.149640    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.149704 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.149666    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 12:10:17.150296 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.150283    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 12:10:17.150365 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.150308    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 12:10:17.150365 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.150322    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeHasSufficientPID"
Jul 23 12:10:17.158206 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.158189    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f91c1b6e4cf3bd4f145a222311c15784-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal\" (UID: \"f91c1b6e4cf3bd4f145a222311c15784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.158270 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.158213    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/05afcdaf9e6abfaa255292cf98e82059-config\") pod \"kube-apiserver-proxy-ip-10-0-137-16.ec2.internal\" (UID: \"05afcdaf9e6abfaa255292cf98e82059\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.158270 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.158229    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f91c1b6e4cf3bd4f145a222311c15784-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal\" (UID: \"f91c1b6e4cf3bd4f145a222311c15784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.183750 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.183724    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-16.ec2.internal\" not found" node="ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.188412 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.188395    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-16.ec2.internal\" not found" node="ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.216778 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.216731    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.259113 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.259082    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f91c1b6e4cf3bd4f145a222311c15784-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal\" (UID: \"f91c1b6e4cf3bd4f145a222311c15784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.259241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.259122    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f91c1b6e4cf3bd4f145a222311c15784-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal\" (UID: \"f91c1b6e4cf3bd4f145a222311c15784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.259241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.259149    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/05afcdaf9e6abfaa255292cf98e82059-config\") pod \"kube-apiserver-proxy-ip-10-0-137-16.ec2.internal\" (UID: \"05afcdaf9e6abfaa255292cf98e82059\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.259241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.259186    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f91c1b6e4cf3bd4f145a222311c15784-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal\" (UID: \"f91c1b6e4cf3bd4f145a222311c15784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.259241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.259213    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/05afcdaf9e6abfaa255292cf98e82059-config\") pod \"kube-apiserver-proxy-ip-10-0-137-16.ec2.internal\" (UID: \"05afcdaf9e6abfaa255292cf98e82059\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.259241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.259216    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f91c1b6e4cf3bd4f145a222311c15784-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal\" (UID: \"f91c1b6e4cf3bd4f145a222311c15784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.317011 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.316922    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.419130 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.419087    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.488465 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.488419    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.491347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.491318    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:17.519959 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.519928    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.623349 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.623257    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.724442 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.724410    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.778709 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.778671    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 12:10:17.779339 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.778863    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 12:10:17.827045 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.827009    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.867185 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.867147    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 12:05:16 +0000 UTC" deadline="2028-05-01 12:47:39.556576826 +0000 UTC"
Jul 23 12:10:17.867185 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.867178    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15552h37m21.689401789s"
Jul 23 12:10:17.927814 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:17.927714    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:17.972422 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:17.972392    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 12:10:18.028607 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.028570    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:18.133923 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.133869    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:18.184743 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.184686    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 12:10:18.235920 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.235884    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-16.ec2.internal\" not found"
Jul 23 12:10:18.245285 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.245265    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 12:10:18.254927 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.254900    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:18.269324 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.269305    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 12:10:18.270445 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.270431    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal"
Jul 23 12:10:18.280200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.280182    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 12:10:18.383741 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:18.383710    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf91c1b6e4cf3bd4f145a222311c15784.slice/crio-a5657387ed4f2b48f647535f9cce509d9601f226c39c86883e7707401e5cd6d4 WatchSource:0}: Error finding container a5657387ed4f2b48f647535f9cce509d9601f226c39c86883e7707401e5cd6d4: Status 404 returned error can't find the container with id a5657387ed4f2b48f647535f9cce509d9601f226c39c86883e7707401e5cd6d4
Jul 23 12:10:18.389796 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.389768    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 12:10:18.474625 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:18.474598    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05afcdaf9e6abfaa255292cf98e82059.slice/crio-1374682ab35007efbb3ec1d099f84159061d4e8c37105b201414553d594e98a0 WatchSource:0}: Error finding container 1374682ab35007efbb3ec1d099f84159061d4e8c37105b201414553d594e98a0: Status 404 returned error can't find the container with id 1374682ab35007efbb3ec1d099f84159061d4e8c37105b201414553d594e98a0
Jul 23 12:10:18.698326 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.698292    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 12:10:18.828181 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.828095    2565 apiserver.go:52] "Watching apiserver"
Jul 23 12:10:18.839272 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.839247    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 12:10:18.839650 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.839625    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-7wjgn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26","openshift-cluster-node-tuning-operator/tuned-ncfx5","openshift-image-registry/node-ca-9dtwz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal","openshift-multus/multus-m25tq","openshift-multus/network-metrics-daemon-xb7r6","openshift-network-diagnostics/network-check-target-tkw2s","kube-system/global-pull-secret-syncer-f22wm","kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal","openshift-dns/node-resolver-89d2f","openshift-multus/multus-additional-cni-plugins-mrgwq","openshift-network-operator/iptables-alerter-vwrkm","openshift-ovn-kubernetes/ovnkube-node-tpwhl"]
Jul 23 12:10:18.841604 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.841585    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:18.841689 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.841666    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:18.843292 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.843268    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.845156 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.845063    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.846417 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.846399    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.846892 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.846761    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5gvz5\""
Jul 23 12:10:18.846892 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.846786    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 12:10:18.846892 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.846797    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 12:10:18.847841 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.847818    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.848243 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.848206    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jzs64\""
Jul 23 12:10:18.848243 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.848206    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 12:10:18.848243 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.848206    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 12:10:18.849330 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.849312    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.849437 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.849405    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 12:10:18.850945 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.850928    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:18.851836 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.851472    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:18.854512 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.854475    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.855193 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.854974    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 12:10:18.855397 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855345    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 12:10:18.855397 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855361    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.855818 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855623    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 12:10:18.855818 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855659    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 12:10:18.855818 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855683    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 12:10:18.855818 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855660    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 12:10:18.855818 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855711    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 12:10:18.855818 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855720    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 12:10:18.856110 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855895    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 12:10:18.856110 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855928    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sqjlk\""
Jul 23 12:10:18.856110 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855931    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 12:10:18.856110 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.855976    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-67r85\""
Jul 23 12:10:18.856110 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.856088    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 12:10:18.856110 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.856102    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dphzz\""
Jul 23 12:10:18.856705 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.856685    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 12:10:18.857081 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.857030    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 12:10:18.857253 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.857237    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:18.857342 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.857325    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:18.858607 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.858586    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:18.860139 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.860122    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:18.863629 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.863596    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 12:10:18.863722 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.863645    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 12:10:18.863890 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.863873    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 12:10:18.867944 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.867922    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 12:10:18.868039 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.867980    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 12:10:18.868130 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.868109    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sjpkj\""
Jul 23 12:10:18.868130 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.868124    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 12:10:18.868289 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.868205    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 12:10:18.868289 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.868263    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5v6nm\""
Jul 23 12:10:18.868395 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.868315    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 12:10:18.868395 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.868389    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 12:10:18.868771 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.868736    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bnwb8\""
Jul 23 12:10:18.869404 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.869325    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 12:10:18.869493 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.869333    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zk7gx\""
Jul 23 12:10:18.870015 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.869991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnkf8\" (UniqueName: \"kubernetes.io/projected/e6fa77ef-d668-4a5a-a79a-46bd992b1029-kube-api-access-xnkf8\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.870081 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870036    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-modprobe-d\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.870081 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870062    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/57b1679f-ad45-4825-817a-69cc071dfa40-etc-tuned\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.870171 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870087    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-run-ovn\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.870171 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870113    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-etc-selinux\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.870171 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870140    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-sys-fs\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.870308 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870173    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-cnibin\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.870308 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870210    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-conf-dir\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.870308 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870235    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs9vl\" (UniqueName: \"kubernetes.io/projected/ccd0a997-8423-410e-bbda-4cc49e6c94d9-kube-api-access-cs9vl\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:18.870308 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-sysctl-d\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.870497 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870304    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e6fa77ef-d668-4a5a-a79a-46bd992b1029-ovnkube-config\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.870497 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870340    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfb4z\" (UniqueName: \"kubernetes.io/projected/d387660f-5d09-497c-8aa3-16397dd07da7-kube-api-access-pfb4z\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.870497 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870370    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-run-multus-certs\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.870497 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870419    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa53ff2f-84ce-4767-8e00-a0230504134b-dbus\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:18.870497 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870445    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9wtt\" (UniqueName: \"kubernetes.io/projected/c24d092b-c557-43b9-a1ce-dd4d7eb8f55c-kube-api-access-p9wtt\") pod \"node-resolver-89d2f\" (UID: \"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c\") " pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.870497 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870475    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-run\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870502    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-run-netns\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870527    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52ac0e67-2df0-4d14-84b1-06ba64808ccf-host\") pod \"node-ca-9dtwz\" (UID: \"52ac0e67-2df0-4d14-84b1-06ba64808ccf\") " pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870551    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-system-cni-dir\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870574    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-kubelet\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870597    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa53ff2f-84ce-4767-8e00-a0230504134b-kubelet-config\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870636    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-run-systemd\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870666    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e6fa77ef-d668-4a5a-a79a-46bd992b1029-env-overrides\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870718    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-hostroot\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870745    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vncjz\" (UniqueName: \"kubernetes.io/projected/3cab005d-7ad9-469f-85ea-a83ec7f0f072-kube-api-access-vncjz\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.870814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870797    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfd8b\" (UniqueName: \"kubernetes.io/projected/57b1679f-ad45-4825-817a-69cc071dfa40-kube-api-access-nfd8b\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870824    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e6fa77ef-d668-4a5a-a79a-46bd992b1029-ovn-node-metrics-cert\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e6fa77ef-d668-4a5a-a79a-46bd992b1029-ovnkube-script-lib\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870900    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-sysconfig\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-socket-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.870981    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-var-lib-cni-bin\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871017    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-sysctl-conf\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871037    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f6277871-5e9e-4f3b-bd37-248738e0c101-iptables-alerter-script\") pod \"iptables-alerter-vwrkm\" (UID: \"f6277871-5e9e-4f3b-bd37-248738e0c101\") " pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871058    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6277871-5e9e-4f3b-bd37-248738e0c101-host-slash\") pod \"iptables-alerter-vwrkm\" (UID: \"f6277871-5e9e-4f3b-bd37-248738e0c101\") " pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871095    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-kubernetes\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.871144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871127    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-slash\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871156    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-etc-openvswitch\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871186    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-run-openvswitch\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871216    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52ac0e67-2df0-4d14-84b1-06ba64808ccf-serviceca\") pod \"node-ca-9dtwz\" (UID: \"52ac0e67-2df0-4d14-84b1-06ba64808ccf\") " pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871243    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c24d092b-c557-43b9-a1ce-dd4d7eb8f55c-hosts-file\") pod \"node-resolver-89d2f\" (UID: \"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c\") " pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871269    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-socket-dir-parent\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-run-netns\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871345    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-lib-modules\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871379    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-host\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871395    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57b1679f-ad45-4825-817a-69cc071dfa40-tmp\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871417    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-var-lib-openvswitch\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871442    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871465    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c24d092b-c557-43b9-a1ce-dd4d7eb8f55c-tmp-dir\") pod \"node-resolver-89d2f\" (UID: \"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c\") " pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871480    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-var-lib-kubelet\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871493    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-systemd-units\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871515    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-log-socket\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.871615 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871543    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871644    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l5kf\" (UniqueName: \"kubernetes.io/projected/52ac0e67-2df0-4d14-84b1-06ba64808ccf-kube-api-access-7l5kf\") pod \"node-ca-9dtwz\" (UID: \"52ac0e67-2df0-4d14-84b1-06ba64808ccf\") " pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871675    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-cni-dir\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871699    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-run-k8s-cni-cncf-io\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871722    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-etc-kubernetes\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-node-log\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-run-ovn-kubernetes\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871824    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871853    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-device-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871901    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871923    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-var-lib-cni-multus\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-var-lib-kubelet\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.871981    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cab005d-7ad9-469f-85ea-a83ec7f0f072-cni-binary-copy\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.872003    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-systemd\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.872040    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-sys\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.872468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.872065    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-cni-netd\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.873343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.872088    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pm8h\" (UniqueName: \"kubernetes.io/projected/f6277871-5e9e-4f3b-bd37-248738e0c101-kube-api-access-4pm8h\") pod \"iptables-alerter-vwrkm\" (UID: \"f6277871-5e9e-4f3b-bd37-248738e0c101\") " pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.873343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.872130    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-registration-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.873343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.872166    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-os-release\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.873343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.872220    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-daemon-config\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.873343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.872244    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-cni-bin\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.882951 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.882927    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 12:10:18.956020 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.955992    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 12:10:18.972718 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972684    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwjxc\" (UniqueName: \"kubernetes.io/projected/17efe563-dcf8-4bf3-9c27-dbdecf122831-kube-api-access-kwjxc\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:18.972886 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-system-cni-dir\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.972886 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972796    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-kubelet\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.972886 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972835    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-system-cni-dir\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:18.972886 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972856    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa53ff2f-84ce-4767-8e00-a0230504134b-kubelet-config\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:18.972886 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972874    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-run-systemd\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e6fa77ef-d668-4a5a-a79a-46bd992b1029-env-overrides\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972913    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-hostroot\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972916    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-kubelet\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972931    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-run-systemd\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972937    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vncjz\" (UniqueName: \"kubernetes.io/projected/3cab005d-7ad9-469f-85ea-a83ec7f0f072-kube-api-access-vncjz\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972981    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfd8b\" (UniqueName: \"kubernetes.io/projected/57b1679f-ad45-4825-817a-69cc071dfa40-kube-api-access-nfd8b\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973007    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e6fa77ef-d668-4a5a-a79a-46bd992b1029-ovn-node-metrics-cert\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973030    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e6fa77ef-d668-4a5a-a79a-46bd992b1029-ovnkube-script-lib\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973056    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-cnibin\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.972869    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-system-cni-dir\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.973124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973082    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/17efe563-dcf8-4bf3-9c27-dbdecf122831-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973146    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa53ff2f-84ce-4767-8e00-a0230504134b-kubelet-config\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973197    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-sysconfig\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973233    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-socket-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973240    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-hostroot\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-var-lib-cni-bin\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973282    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-sysctl-conf\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973293    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-sysconfig\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973307    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f6277871-5e9e-4f3b-bd37-248738e0c101-iptables-alerter-script\") pod \"iptables-alerter-vwrkm\" (UID: \"f6277871-5e9e-4f3b-bd37-248738e0c101\") " pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973332    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-var-lib-cni-bin\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973333    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6277871-5e9e-4f3b-bd37-248738e0c101-host-slash\") pod \"iptables-alerter-vwrkm\" (UID: \"f6277871-5e9e-4f3b-bd37-248738e0c101\") " pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973372    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-os-release\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973377    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6277871-5e9e-4f3b-bd37-248738e0c101-host-slash\") pod \"iptables-alerter-vwrkm\" (UID: \"f6277871-5e9e-4f3b-bd37-248738e0c101\") " pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973410    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-kubernetes\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973418    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-socket-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973435    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-slash\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-etc-openvswitch\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973483    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-run-openvswitch\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.973634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973474    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52ac0e67-2df0-4d14-84b1-06ba64808ccf-serviceca\") pod \"node-ca-9dtwz\" (UID: \"52ac0e67-2df0-4d14-84b1-06ba64808ccf\") " pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973520    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-sysctl-conf\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973535    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973565    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-etc-openvswitch\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973572    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e6fa77ef-d668-4a5a-a79a-46bd992b1029-env-overrides\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973566    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c24d092b-c557-43b9-a1ce-dd4d7eb8f55c-hosts-file\") pod \"node-resolver-89d2f\" (UID: \"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c\") " pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973643    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-socket-dir-parent\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973612    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c24d092b-c557-43b9-a1ce-dd4d7eb8f55c-hosts-file\") pod \"node-resolver-89d2f\" (UID: \"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c\") " pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973646    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-slash\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973673    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-run-openvswitch\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973679    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-run-netns\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973611    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-kubernetes\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973702    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e6fa77ef-d668-4a5a-a79a-46bd992b1029-ovnkube-script-lib\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973722    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-socket-dir-parent\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973707    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-lib-modules\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973757    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-run-netns\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-host\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.974564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973808    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57b1679f-ad45-4825-817a-69cc071dfa40-tmp\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-var-lib-openvswitch\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973855    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-lib-modules\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973859    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/17efe563-dcf8-4bf3-9c27-dbdecf122831-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973863    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-host\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973888    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973927    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c24d092b-c557-43b9-a1ce-dd4d7eb8f55c-tmp-dir\") pod \"node-resolver-89d2f\" (UID: \"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c\") " pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973960    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-var-lib-kubelet\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.974004    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974003    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-systemd-units\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974015    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-var-lib-kubelet\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974038    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-log-socket\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974048    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-systemd-units\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974008    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f6277871-5e9e-4f3b-bd37-248738e0c101-iptables-alerter-script\") pod \"iptables-alerter-vwrkm\" (UID: \"f6277871-5e9e-4f3b-bd37-248738e0c101\") " pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.974075    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs podName:ccd0a997-8423-410e-bbda-4cc49e6c94d9 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:19.474055066 +0000 UTC m=+3.129563867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs") pod "network-metrics-daemon-xb7r6" (UID: "ccd0a997-8423-410e-bbda-4cc49e6c94d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974079    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52ac0e67-2df0-4d14-84b1-06ba64808ccf-serviceca\") pod \"node-ca-9dtwz\" (UID: \"52ac0e67-2df0-4d14-84b1-06ba64808ccf\") " pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.973911    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-var-lib-openvswitch\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.975430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974081    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-log-socket\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974093    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974129    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974129    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l5kf\" (UniqueName: \"kubernetes.io/projected/52ac0e67-2df0-4d14-84b1-06ba64808ccf-kube-api-access-7l5kf\") pod \"node-ca-9dtwz\" (UID: \"52ac0e67-2df0-4d14-84b1-06ba64808ccf\") " pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974165    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-cni-dir\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974190    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-run-k8s-cni-cncf-io\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974215    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-etc-kubernetes\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974240    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-node-log\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974269    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c24d092b-c557-43b9-a1ce-dd4d7eb8f55c-tmp-dir\") pod \"node-resolver-89d2f\" (UID: \"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c\") " pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974281    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-run-k8s-cni-cncf-io\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974285    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-node-log\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974307    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-run-ovn-kubernetes\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974331    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-etc-kubernetes\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974377    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-run-ovn-kubernetes\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974391    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-device-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974425    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-cni-dir\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.976347 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974445    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974469    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974485    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974493    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-var-lib-cni-multus\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974500    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-device-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-var-lib-kubelet\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974543    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cab005d-7ad9-469f-85ea-a83ec7f0f072-cni-binary-copy\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.974565    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-systemd\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.974626    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret podName:aa53ff2f-84ce-4767-8e00-a0230504134b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:19.474607486 +0000 UTC m=+3.130116269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret") pod "global-pull-secret-syncer-f22wm" (UID: "aa53ff2f-84ce-4767-8e00-a0230504134b") : object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974565    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-systemd\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974658    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-var-lib-cni-multus\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974661    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-sys\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974693    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-var-lib-kubelet\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974696    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-cni-netd\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974723    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pm8h\" (UniqueName: \"kubernetes.io/projected/f6277871-5e9e-4f3b-bd37-248738e0c101-kube-api-access-4pm8h\") pod \"iptables-alerter-vwrkm\" (UID: \"f6277871-5e9e-4f3b-bd37-248738e0c101\") " pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974737    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-sys\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.977216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974755    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-registration-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974799    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-os-release\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974811    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-cni-netd\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974824    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-daemon-config\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974847    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-cni-bin\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974862    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-registration-dir\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974873    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnkf8\" (UniqueName: \"kubernetes.io/projected/e6fa77ef-d668-4a5a-a79a-46bd992b1029-kube-api-access-xnkf8\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/44b339ea-0da4-414b-b554-25c5eb99bef7-agent-certs\") pod \"konnectivity-agent-7wjgn\" (UID: \"44b339ea-0da4-414b-b554-25c5eb99bef7\") " pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974929    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-modprobe-d\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974952    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/57b1679f-ad45-4825-817a-69cc071dfa40-etc-tuned\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974978    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-run-ovn\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.974973    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-cni-bin\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975005    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-etc-selinux\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975043    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-sys-fs\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975073    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-cnibin\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975100    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-conf-dir\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-etc-selinux\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.977804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs9vl\" (UniqueName: \"kubernetes.io/projected/ccd0a997-8423-410e-bbda-4cc49e6c94d9-kube-api-access-cs9vl\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975154    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-sysctl-d\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975179    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e6fa77ef-d668-4a5a-a79a-46bd992b1029-ovnkube-config\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975187    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cab005d-7ad9-469f-85ea-a83ec7f0f072-cni-binary-copy\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975206    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfb4z\" (UniqueName: \"kubernetes.io/projected/d387660f-5d09-497c-8aa3-16397dd07da7-kube-api-access-pfb4z\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975217    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-run-ovn\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975238    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/44b339ea-0da4-414b-b554-25c5eb99bef7-konnectivity-ca\") pod \"konnectivity-agent-7wjgn\" (UID: \"44b339ea-0da4-414b-b554-25c5eb99bef7\") " pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975267    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-run-multus-certs\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975270    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d387660f-5d09-497c-8aa3-16397dd07da7-sys-fs\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975293    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-os-release\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975304    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17efe563-dcf8-4bf3-9c27-dbdecf122831-cni-binary-copy\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975318    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-host-run-multus-certs\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975323    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-sysctl-d\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975334    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa53ff2f-84ce-4767-8e00-a0230504134b-dbus\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975362    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9wtt\" (UniqueName: \"kubernetes.io/projected/c24d092b-c557-43b9-a1ce-dd4d7eb8f55c-kube-api-access-p9wtt\") pod \"node-resolver-89d2f\" (UID: \"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c\") " pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975371    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-etc-modprobe-d\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975386    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-run\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975410    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-run-netns\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.978290 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52ac0e67-2df0-4d14-84b1-06ba64808ccf-host\") pod \"node-ca-9dtwz\" (UID: \"52ac0e67-2df0-4d14-84b1-06ba64808ccf\") " pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975511    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52ac0e67-2df0-4d14-84b1-06ba64808ccf-host\") pod \"node-ca-9dtwz\" (UID: \"52ac0e67-2df0-4d14-84b1-06ba64808ccf\") " pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975558    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-cnibin\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975661    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa53ff2f-84ce-4767-8e00-a0230504134b-dbus\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975762    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/57b1679f-ad45-4825-817a-69cc071dfa40-run\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975819    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6fa77ef-d668-4a5a-a79a-46bd992b1029-host-run-netns\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975812    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e6fa77ef-d668-4a5a-a79a-46bd992b1029-ovnkube-config\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975887    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-conf-dir\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.975916    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3cab005d-7ad9-469f-85ea-a83ec7f0f072-multus-daemon-config\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.977424    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57b1679f-ad45-4825-817a-69cc071dfa40-tmp\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.977435    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e6fa77ef-d668-4a5a-a79a-46bd992b1029-ovn-node-metrics-cert\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.978910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.977594    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/57b1679f-ad45-4825-817a-69cc071dfa40-etc-tuned\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.986405 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.986379    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 12:10:18.986405 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.986407    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 12:10:18.986587 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.986421    2565 projected.go:196] Error preparing data for projected volume kube-api-access-t7vgf for pod openshift-network-diagnostics/network-check-target-tkw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:18.986587 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.986453    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfb4z\" (UniqueName: \"kubernetes.io/projected/d387660f-5d09-497c-8aa3-16397dd07da7-kube-api-access-pfb4z\") pod \"aws-ebs-csi-driver-node-sjh26\" (UID: \"d387660f-5d09-497c-8aa3-16397dd07da7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:18.986587 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:18.986493    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf podName:501c1bcd-51e9-4cc3-9356-ab8ed8de7a87 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:19.486472937 +0000 UTC m=+3.141981719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t7vgf" (UniqueName: "kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf") pod "network-check-target-tkw2s" (UID: "501c1bcd-51e9-4cc3-9356-ab8ed8de7a87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:18.988377 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.988351    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9wtt\" (UniqueName: \"kubernetes.io/projected/c24d092b-c557-43b9-a1ce-dd4d7eb8f55c-kube-api-access-p9wtt\") pod \"node-resolver-89d2f\" (UID: \"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c\") " pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:18.988748 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.988716    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l5kf\" (UniqueName: \"kubernetes.io/projected/52ac0e67-2df0-4d14-84b1-06ba64808ccf-kube-api-access-7l5kf\") pod \"node-ca-9dtwz\" (UID: \"52ac0e67-2df0-4d14-84b1-06ba64808ccf\") " pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:18.989242 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.989218    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pm8h\" (UniqueName: \"kubernetes.io/projected/f6277871-5e9e-4f3b-bd37-248738e0c101-kube-api-access-4pm8h\") pod \"iptables-alerter-vwrkm\" (UID: \"f6277871-5e9e-4f3b-bd37-248738e0c101\") " pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:18.989751 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.989726    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vncjz\" (UniqueName: \"kubernetes.io/projected/3cab005d-7ad9-469f-85ea-a83ec7f0f072-kube-api-access-vncjz\") pod \"multus-m25tq\" (UID: \"3cab005d-7ad9-469f-85ea-a83ec7f0f072\") " pod="openshift-multus/multus-m25tq"
Jul 23 12:10:18.990019 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.989996    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfd8b\" (UniqueName: \"kubernetes.io/projected/57b1679f-ad45-4825-817a-69cc071dfa40-kube-api-access-nfd8b\") pod \"tuned-ncfx5\" (UID: \"57b1679f-ad45-4825-817a-69cc071dfa40\") " pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:18.990308 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.990286    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnkf8\" (UniqueName: \"kubernetes.io/projected/e6fa77ef-d668-4a5a-a79a-46bd992b1029-kube-api-access-xnkf8\") pod \"ovnkube-node-tpwhl\" (UID: \"e6fa77ef-d668-4a5a-a79a-46bd992b1029\") " pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:18.990725 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:18.990709    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs9vl\" (UniqueName: \"kubernetes.io/projected/ccd0a997-8423-410e-bbda-4cc49e6c94d9-kube-api-access-cs9vl\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:19.019345 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.019302    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xhcw4"
Jul 23 12:10:19.029216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.029175    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xhcw4"
Jul 23 12:10:19.048702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.048642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal" event={"ID":"05afcdaf9e6abfaa255292cf98e82059","Type":"ContainerStarted","Data":"1374682ab35007efbb3ec1d099f84159061d4e8c37105b201414553d594e98a0"}
Jul 23 12:10:19.049902 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.049870    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal" event={"ID":"f91c1b6e4cf3bd4f145a222311c15784","Type":"ContainerStarted","Data":"a5657387ed4f2b48f647535f9cce509d9601f226c39c86883e7707401e5cd6d4"}
Jul 23 12:10:19.076317 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076289    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/17efe563-dcf8-4bf3-9c27-dbdecf122831-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.076499 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076371    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/44b339ea-0da4-414b-b554-25c5eb99bef7-agent-certs\") pod \"konnectivity-agent-7wjgn\" (UID: \"44b339ea-0da4-414b-b554-25c5eb99bef7\") " pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:19.076499 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076406    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/44b339ea-0da4-414b-b554-25c5eb99bef7-konnectivity-ca\") pod \"konnectivity-agent-7wjgn\" (UID: \"44b339ea-0da4-414b-b554-25c5eb99bef7\") " pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:19.076499 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076427    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17efe563-dcf8-4bf3-9c27-dbdecf122831-cni-binary-copy\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.076499 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076453    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwjxc\" (UniqueName: \"kubernetes.io/projected/17efe563-dcf8-4bf3-9c27-dbdecf122831-kube-api-access-kwjxc\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.076499 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076477    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-system-cni-dir\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.076764 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076534    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-system-cni-dir\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.076764 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076570    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-cnibin\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.076764 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076597    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/17efe563-dcf8-4bf3-9c27-dbdecf122831-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.076764 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076626    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-os-release\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.076764 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076656    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.077014 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076857    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.077014 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076913    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-cnibin\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.077014 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.076950    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17efe563-dcf8-4bf3-9c27-dbdecf122831-os-release\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.077151 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.077025    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/17efe563-dcf8-4bf3-9c27-dbdecf122831-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.077151 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.077058    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/44b339ea-0da4-414b-b554-25c5eb99bef7-konnectivity-ca\") pod \"konnectivity-agent-7wjgn\" (UID: \"44b339ea-0da4-414b-b554-25c5eb99bef7\") " pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:19.077151 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.077071    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17efe563-dcf8-4bf3-9c27-dbdecf122831-cni-binary-copy\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.077253 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.077191    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/17efe563-dcf8-4bf3-9c27-dbdecf122831-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.079279 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.079225    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/44b339ea-0da4-414b-b554-25c5eb99bef7-agent-certs\") pod \"konnectivity-agent-7wjgn\" (UID: \"44b339ea-0da4-414b-b554-25c5eb99bef7\") " pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:19.086393 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.086374    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwjxc\" (UniqueName: \"kubernetes.io/projected/17efe563-dcf8-4bf3-9c27-dbdecf122831-kube-api-access-kwjxc\") pod \"multus-additional-cni-plugins-mrgwq\" (UID: \"17efe563-dcf8-4bf3-9c27-dbdecf122831\") " pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.158906 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.158873    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-89d2f"
Jul 23 12:10:19.166981 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.166956    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ncfx5"
Jul 23 12:10:19.175827 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.175804    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9dtwz"
Jul 23 12:10:19.180636 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.180614    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:19.186555 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.186532    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m25tq"
Jul 23 12:10:19.195789 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.195750    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vwrkm"
Jul 23 12:10:19.202801 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.202779    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26"
Jul 23 12:10:19.209557 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.209496    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:19.215338 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.215317    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mrgwq"
Jul 23 12:10:19.456379 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:19.456356    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b339ea_0da4_414b_b554_25c5eb99bef7.slice/crio-7647084584c1eef0deae70c9ee7fe3648eecdc154dd6c6a099d2b904df68d2ad WatchSource:0}: Error finding container 7647084584c1eef0deae70c9ee7fe3648eecdc154dd6c6a099d2b904df68d2ad: Status 404 returned error can't find the container with id 7647084584c1eef0deae70c9ee7fe3648eecdc154dd6c6a099d2b904df68d2ad
Jul 23 12:10:19.470195 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:19.470166    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17efe563_dcf8_4bf3_9c27_dbdecf122831.slice/crio-520e292c964ddc1fb3e314d5dfb0f486f62ecd2b4422203d21a7013f0be6918d WatchSource:0}: Error finding container 520e292c964ddc1fb3e314d5dfb0f486f62ecd2b4422203d21a7013f0be6918d: Status 404 returned error can't find the container with id 520e292c964ddc1fb3e314d5dfb0f486f62ecd2b4422203d21a7013f0be6918d
Jul 23 12:10:19.475025 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:19.475000    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6277871_5e9e_4f3b_bd37_248738e0c101.slice/crio-c1631922e47b3241116d1e38e802c7f21bfe800f658c9b6e3a6a00873c1dfd4a WatchSource:0}: Error finding container c1631922e47b3241116d1e38e802c7f21bfe800f658c9b6e3a6a00873c1dfd4a: Status 404 returned error can't find the container with id c1631922e47b3241116d1e38e802c7f21bfe800f658c9b6e3a6a00873c1dfd4a
Jul 23 12:10:19.475469 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:19.475449    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc24d092b_c557_43b9_a1ce_dd4d7eb8f55c.slice/crio-f038798b6b77df883baf5f5cd324aa5a1acb174ab89a56a53fe12362cf272cc8 WatchSource:0}: Error finding container f038798b6b77df883baf5f5cd324aa5a1acb174ab89a56a53fe12362cf272cc8: Status 404 returned error can't find the container with id f038798b6b77df883baf5f5cd324aa5a1acb174ab89a56a53fe12362cf272cc8
Jul 23 12:10:19.476297 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:19.476273    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cab005d_7ad9_469f_85ea_a83ec7f0f072.slice/crio-08e6c3ab0df78d452b23261d5022a34edb0097562d5d8b606de2c6f5f64ad9ef WatchSource:0}: Error finding container 08e6c3ab0df78d452b23261d5022a34edb0097562d5d8b606de2c6f5f64ad9ef: Status 404 returned error can't find the container with id 08e6c3ab0df78d452b23261d5022a34edb0097562d5d8b606de2c6f5f64ad9ef
Jul 23 12:10:19.477329 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:19.477305    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b1679f_ad45_4825_817a_69cc071dfa40.slice/crio-5f2b51f3eb29cc53f5419e86fe6154e396822edfef48e323eadc9fd7fa4560ca WatchSource:0}: Error finding container 5f2b51f3eb29cc53f5419e86fe6154e396822edfef48e323eadc9fd7fa4560ca: Status 404 returned error can't find the container with id 5f2b51f3eb29cc53f5419e86fe6154e396822edfef48e323eadc9fd7fa4560ca
Jul 23 12:10:19.479096 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:19.478265    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6fa77ef_d668_4a5a_a79a_46bd992b1029.slice/crio-e02b14e7ea1d5cf74f00855c213ac23af0eda2ca560b998d988274baa874dbe5 WatchSource:0}: Error finding container e02b14e7ea1d5cf74f00855c213ac23af0eda2ca560b998d988274baa874dbe5: Status 404 returned error can't find the container with id e02b14e7ea1d5cf74f00855c213ac23af0eda2ca560b998d988274baa874dbe5
Jul 23 12:10:19.479096 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.478912    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:19.479096 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.478958    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:19.479096 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:19.479053    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:19.479096 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:19.479060    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:19.479713 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:19.479107    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret podName:aa53ff2f-84ce-4767-8e00-a0230504134b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:20.479089646 +0000 UTC m=+4.134598447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret") pod "global-pull-secret-syncer-f22wm" (UID: "aa53ff2f-84ce-4767-8e00-a0230504134b") : object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:19.479713 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:19.479125    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs podName:ccd0a997-8423-410e-bbda-4cc49e6c94d9 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:20.479116234 +0000 UTC m=+4.134625030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs") pod "network-metrics-daemon-xb7r6" (UID: "ccd0a997-8423-410e-bbda-4cc49e6c94d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:19.479713 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:19.479413    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd387660f_5d09_497c_8aa3_16397dd07da7.slice/crio-80561b963591943884312bec2de3344bc4693c1dfbf70a2f372a743c82e76055 WatchSource:0}: Error finding container 80561b963591943884312bec2de3344bc4693c1dfbf70a2f372a743c82e76055: Status 404 returned error can't find the container with id 80561b963591943884312bec2de3344bc4693c1dfbf70a2f372a743c82e76055
Jul 23 12:10:19.480133 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:19.480116    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ac0e67_2df0_4d14_84b1_06ba64808ccf.slice/crio-343b86506ccfa93c158ff2cd59ecc11e06dee7d7b6851985b5128463a8b4fc67 WatchSource:0}: Error finding container 343b86506ccfa93c158ff2cd59ecc11e06dee7d7b6851985b5128463a8b4fc67: Status 404 returned error can't find the container with id 343b86506ccfa93c158ff2cd59ecc11e06dee7d7b6851985b5128463a8b4fc67
Jul 23 12:10:19.580081 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:19.580054    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:19.580235 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:19.580187    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 12:10:19.580235 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:19.580201    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 12:10:19.580235 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:19.580210    2565 projected.go:196] Error preparing data for projected volume kube-api-access-t7vgf for pod openshift-network-diagnostics/network-check-target-tkw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:19.580353 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:19.580250    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf podName:501c1bcd-51e9-4cc3-9356-ab8ed8de7a87 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:20.580238961 +0000 UTC m=+4.235747743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t7vgf" (UniqueName: "kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf") pod "network-check-target-tkw2s" (UID: "501c1bcd-51e9-4cc3-9356-ab8ed8de7a87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:20.033684 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.033303    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 12:05:19 +0000 UTC" deadline="2028-03-16 17:27:18.232426533 +0000 UTC"
Jul 23 12:10:20.033684 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.033565    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14453h16m58.198869574s"
Jul 23 12:10:20.043793 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.043615    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:20.043793 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.043771    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:20.044225 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.044077    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:20.044225 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.044177    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:20.061037 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.060957    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9dtwz" event={"ID":"52ac0e67-2df0-4d14-84b1-06ba64808ccf","Type":"ContainerStarted","Data":"343b86506ccfa93c158ff2cd59ecc11e06dee7d7b6851985b5128463a8b4fc67"}
Jul 23 12:10:20.072635 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.072599    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26" event={"ID":"d387660f-5d09-497c-8aa3-16397dd07da7","Type":"ContainerStarted","Data":"80561b963591943884312bec2de3344bc4693c1dfbf70a2f372a743c82e76055"}
Jul 23 12:10:20.087824 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.087653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" event={"ID":"e6fa77ef-d668-4a5a-a79a-46bd992b1029","Type":"ContainerStarted","Data":"e02b14e7ea1d5cf74f00855c213ac23af0eda2ca560b998d988274baa874dbe5"}
Jul 23 12:10:20.093109 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.093064    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ncfx5" event={"ID":"57b1679f-ad45-4825-817a-69cc071dfa40","Type":"ContainerStarted","Data":"5f2b51f3eb29cc53f5419e86fe6154e396822edfef48e323eadc9fd7fa4560ca"}
Jul 23 12:10:20.098032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.098002    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m25tq" event={"ID":"3cab005d-7ad9-469f-85ea-a83ec7f0f072","Type":"ContainerStarted","Data":"08e6c3ab0df78d452b23261d5022a34edb0097562d5d8b606de2c6f5f64ad9ef"}
Jul 23 12:10:20.103277 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.103250    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrgwq" event={"ID":"17efe563-dcf8-4bf3-9c27-dbdecf122831","Type":"ContainerStarted","Data":"520e292c964ddc1fb3e314d5dfb0f486f62ecd2b4422203d21a7013f0be6918d"}
Jul 23 12:10:20.110962 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.110709    2565 generic.go:356] "Generic (PLEG): container finished" podID="f91c1b6e4cf3bd4f145a222311c15784" containerID="0b23ae32df0f94ca0898047dc867b5e3fba231e1f062330273fc826ddf3a9b8b" exitCode=0
Jul 23 12:10:20.110962 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.110864    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal" event={"ID":"f91c1b6e4cf3bd4f145a222311c15784","Type":"ContainerDied","Data":"0b23ae32df0f94ca0898047dc867b5e3fba231e1f062330273fc826ddf3a9b8b"}
Jul 23 12:10:20.117421 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.117389    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-89d2f" event={"ID":"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c","Type":"ContainerStarted","Data":"f038798b6b77df883baf5f5cd324aa5a1acb174ab89a56a53fe12362cf272cc8"}
Jul 23 12:10:20.123527 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.123501    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vwrkm" event={"ID":"f6277871-5e9e-4f3b-bd37-248738e0c101","Type":"ContainerStarted","Data":"c1631922e47b3241116d1e38e802c7f21bfe800f658c9b6e3a6a00873c1dfd4a"}
Jul 23 12:10:20.136433 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.136368    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7wjgn" event={"ID":"44b339ea-0da4-414b-b554-25c5eb99bef7","Type":"ContainerStarted","Data":"7647084584c1eef0deae70c9ee7fe3648eecdc154dd6c6a099d2b904df68d2ad"}
Jul 23 12:10:20.487787 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.487735    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:20.487964 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.487809    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:20.487964 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.487940    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:20.488073 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.488000    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret podName:aa53ff2f-84ce-4767-8e00-a0230504134b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:22.487983661 +0000 UTC m=+6.143492446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret") pod "global-pull-secret-syncer-f22wm" (UID: "aa53ff2f-84ce-4767-8e00-a0230504134b") : object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:20.488411 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.488394    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:20.488496 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.488443    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs podName:ccd0a997-8423-410e-bbda-4cc49e6c94d9 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:22.488428753 +0000 UTC m=+6.143937541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs") pod "network-metrics-daemon-xb7r6" (UID: "ccd0a997-8423-410e-bbda-4cc49e6c94d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:20.589427 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:20.588752    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:20.589427 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.588932    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 12:10:20.589427 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.588955    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 12:10:20.589427 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.588967    2565 projected.go:196] Error preparing data for projected volume kube-api-access-t7vgf for pod openshift-network-diagnostics/network-check-target-tkw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:20.589427 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:20.589031    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf podName:501c1bcd-51e9-4cc3-9356-ab8ed8de7a87 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:22.589011847 +0000 UTC m=+6.244520633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t7vgf" (UniqueName: "kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf") pod "network-check-target-tkw2s" (UID: "501c1bcd-51e9-4cc3-9356-ab8ed8de7a87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:21.034801 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:21.034677    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 12:05:19 +0000 UTC" deadline="2028-03-20 19:03:56.185723891 +0000 UTC"
Jul 23 12:10:21.034801 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:21.034740    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14550h53m35.150987333s"
Jul 23 12:10:21.046180 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:21.044028    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:21.046180 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:21.044224    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:21.150906 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:21.150869    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal" event={"ID":"f91c1b6e4cf3bd4f145a222311c15784","Type":"ContainerStarted","Data":"baa5e1a8ccd346ca7a4f4a58daf199d36afd592e7609a4a3ca21eaa7e8476adb"}
Jul 23 12:10:22.044361 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:22.044329    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:22.044823 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.044471    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:22.044823 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:22.044503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:22.044823 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.044647    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:22.507422 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:22.507340    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:22.507422 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:22.507394    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:22.507647 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.507537    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:22.507647 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.507646    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret podName:aa53ff2f-84ce-4767-8e00-a0230504134b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:26.507623601 +0000 UTC m=+10.163132390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret") pod "global-pull-secret-syncer-f22wm" (UID: "aa53ff2f-84ce-4767-8e00-a0230504134b") : object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:22.508060 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.508039    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:22.508134 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.508092    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs podName:ccd0a997-8423-410e-bbda-4cc49e6c94d9 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:26.508077993 +0000 UTC m=+10.163586784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs") pod "network-metrics-daemon-xb7r6" (UID: "ccd0a997-8423-410e-bbda-4cc49e6c94d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:22.607864 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:22.607826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:22.608046 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.607994    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 12:10:22.608046 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.608015    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 12:10:22.608046 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.608027    2565 projected.go:196] Error preparing data for projected volume kube-api-access-t7vgf for pod openshift-network-diagnostics/network-check-target-tkw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:22.608225 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:22.608082    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf podName:501c1bcd-51e9-4cc3-9356-ab8ed8de7a87 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:26.608064405 +0000 UTC m=+10.263573191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t7vgf" (UniqueName: "kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf") pod "network-check-target-tkw2s" (UID: "501c1bcd-51e9-4cc3-9356-ab8ed8de7a87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:23.046479 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:23.046436    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:23.046950 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:23.046605    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:24.043996 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:24.043959    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:24.043996 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:24.043979    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:24.044245 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:24.044103    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:24.044245 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:24.044205    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:25.044783 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:25.044731    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:25.045219 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:25.044940    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:26.043778 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:26.043738    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:26.043967 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:26.043738    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:26.043967 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.043896    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:26.043967 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.043925    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:26.541356 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:26.541205    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:26.541356 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:26.541262    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:26.541879 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.541391    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:26.541879 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.541474    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs podName:ccd0a997-8423-410e-bbda-4cc49e6c94d9 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:34.541454911 +0000 UTC m=+18.196963696 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs") pod "network-metrics-daemon-xb7r6" (UID: "ccd0a997-8423-410e-bbda-4cc49e6c94d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:26.541879 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.541485    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:26.541879 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.541545    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret podName:aa53ff2f-84ce-4767-8e00-a0230504134b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:34.541528554 +0000 UTC m=+18.197037337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret") pod "global-pull-secret-syncer-f22wm" (UID: "aa53ff2f-84ce-4767-8e00-a0230504134b") : object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:26.642606 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:26.642542    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:26.642788 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.642740    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 12:10:26.642788 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.642786    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 12:10:26.642910 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.642801    2565 projected.go:196] Error preparing data for projected volume kube-api-access-t7vgf for pod openshift-network-diagnostics/network-check-target-tkw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:26.642910 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:26.642864    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf podName:501c1bcd-51e9-4cc3-9356-ab8ed8de7a87 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:34.642846212 +0000 UTC m=+18.298354995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t7vgf" (UniqueName: "kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf") pod "network-check-target-tkw2s" (UID: "501c1bcd-51e9-4cc3-9356-ab8ed8de7a87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:27.044394 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:27.043902    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:27.044394 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:27.044033    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:28.043900 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:28.043819    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:28.044333 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:28.043819    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:28.044333 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:28.043971    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:28.044333 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:28.044062    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:29.044112 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:29.044077    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:29.044561 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:29.044211    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:30.044286 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:30.044251    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:30.044723 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:30.044370    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:30.044723 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:30.044419    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:30.044723 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:30.044494    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:31.043819 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:31.043782    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:31.044003 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:31.043906    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:32.044474 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:32.044438    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:32.044987 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:32.044496    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:32.044987 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:32.044588    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:32.044987 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:32.044711    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:33.044359 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:33.044324    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:33.044521 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:33.044440    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:34.043580 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:34.043542    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:34.043828 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:34.043591    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:34.043828 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.043673    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:34.043927 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.043836    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:34.601581 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:34.601535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:34.602030 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:34.601606    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:34.602030 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.601691    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:34.602030 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.601726    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:34.602030 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.601784    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs podName:ccd0a997-8423-410e-bbda-4cc49e6c94d9 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:50.601751915 +0000 UTC m=+34.257260702 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs") pod "network-metrics-daemon-xb7r6" (UID: "ccd0a997-8423-410e-bbda-4cc49e6c94d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:34.602030 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.601801    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret podName:aa53ff2f-84ce-4767-8e00-a0230504134b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:50.60179472 +0000 UTC m=+34.257303506 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret") pod "global-pull-secret-syncer-f22wm" (UID: "aa53ff2f-84ce-4767-8e00-a0230504134b") : object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:34.702455 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:34.702421    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:34.702629 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.702563    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 12:10:34.702629 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.702591    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 12:10:34.702629 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.702602    2565 projected.go:196] Error preparing data for projected volume kube-api-access-t7vgf for pod openshift-network-diagnostics/network-check-target-tkw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:34.702779 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:34.702652    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf podName:501c1bcd-51e9-4cc3-9356-ab8ed8de7a87 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:50.702639505 +0000 UTC m=+34.358148287 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t7vgf" (UniqueName: "kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf") pod "network-check-target-tkw2s" (UID: "501c1bcd-51e9-4cc3-9356-ab8ed8de7a87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:35.044488 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:35.044404    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:35.044648 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:35.044533    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:36.043702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:36.043661    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:36.043702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:36.043689    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:36.044210 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:36.043820    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:36.044210 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:36.043933    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:37.044919 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:37.044652    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:37.044919 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:37.044747    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:38.043804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:38.043752    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:38.043980 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:38.043891    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:38.043980 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:38.043932    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:38.044061 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:38.044012    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:39.044737 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.044293    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:39.045247 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:39.044932    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:39.196618 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.196545    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9dtwz" event={"ID":"52ac0e67-2df0-4d14-84b1-06ba64808ccf","Type":"ContainerStarted","Data":"16c164a1947ad6a282d8ddb8760805647382fc395bbbbc604f676173c2234d8e"}
Jul 23 12:10:39.198049 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.198021    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26" event={"ID":"d387660f-5d09-497c-8aa3-16397dd07da7","Type":"ContainerStarted","Data":"20ced5abddd36b25934df54e8ddd86126ef5e80fdd9f254a2b8d985694360a19"}
Jul 23 12:10:39.200723 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.200696    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" event={"ID":"e6fa77ef-d668-4a5a-a79a-46bd992b1029","Type":"ContainerStarted","Data":"3504773dbae1762329d716a742a2e329bd08691eadaa937a5754b8c6a12234ac"}
Jul 23 12:10:39.200853 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.200783    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" event={"ID":"e6fa77ef-d668-4a5a-a79a-46bd992b1029","Type":"ContainerStarted","Data":"bc05f1a4d001da60b02851ddcc6bcfa355f939bf4f2a25b02e802ff4de26d5d6"}
Jul 23 12:10:39.200853 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.200832    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" event={"ID":"e6fa77ef-d668-4a5a-a79a-46bd992b1029","Type":"ContainerStarted","Data":"f695f04c4be050afce991e97585c0daf4611e23c361f9a8248af1eeda5ae3239"}
Jul 23 12:10:39.200941 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.200878    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" event={"ID":"e6fa77ef-d668-4a5a-a79a-46bd992b1029","Type":"ContainerStarted","Data":"6f3b5b0e76f518633e7f450093ec38946fa44f6124ffd4ec1299c28d9fdaf4f4"}
Jul 23 12:10:39.200941 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.200907    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" event={"ID":"e6fa77ef-d668-4a5a-a79a-46bd992b1029","Type":"ContainerStarted","Data":"ab593a9527ecff352766c3c3fef15c4edc22fb094895be69071ac1c420102072"}
Jul 23 12:10:39.202276 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.202250    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ncfx5" event={"ID":"57b1679f-ad45-4825-817a-69cc071dfa40","Type":"ContainerStarted","Data":"aa0866fe12282377c7b30a48dd84a4a0b2f24a94865fd9fc5273bcec1e567da8"}
Jul 23 12:10:39.203731 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.203707    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m25tq" event={"ID":"3cab005d-7ad9-469f-85ea-a83ec7f0f072","Type":"ContainerStarted","Data":"d98e9bf3fe8ba37798d8ac09c7a77b8d3e3f635695dfc862156a8ce4118b021c"}
Jul 23 12:10:39.205241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.205220    2565 generic.go:356] "Generic (PLEG): container finished" podID="17efe563-dcf8-4bf3-9c27-dbdecf122831" containerID="005e7b47e65896b4c61e6ac240822c5b5a91623405ca836b526df8222ad224c2" exitCode=0
Jul 23 12:10:39.205332 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.205269    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrgwq" event={"ID":"17efe563-dcf8-4bf3-9c27-dbdecf122831","Type":"ContainerDied","Data":"005e7b47e65896b4c61e6ac240822c5b5a91623405ca836b526df8222ad224c2"}
Jul 23 12:10:39.206656 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.206634    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-89d2f" event={"ID":"c24d092b-c557-43b9-a1ce-dd4d7eb8f55c","Type":"ContainerStarted","Data":"929d69118c6eaf2b1d15255f864eb3408c7d25164e8f894c1ace4348c84aa3ca"}
Jul 23 12:10:39.208025 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.207984    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7wjgn" event={"ID":"44b339ea-0da4-414b-b554-25c5eb99bef7","Type":"ContainerStarted","Data":"9b35b40a1f53498d231088200b5c9034365825712b45cbe1c2a3e2928e55a287"}
Jul 23 12:10:39.209319 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.209303    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal" event={"ID":"05afcdaf9e6abfaa255292cf98e82059","Type":"ContainerStarted","Data":"70f21a2142535613650252625d2f911999b2906689fcdd72a91d0be41704b07f"}
Jul 23 12:10:39.214464 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.214410    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9dtwz" podStartSLOduration=7.50430712 podStartE2EDuration="22.214388579s" totalImagesPullingTime="14.710081459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:39.213727341 +0000 UTC m=+22.869236148" watchObservedRunningTime="2026-07-23 12:10:39.214388579 +0000 UTC m=+22.869897383"
Jul 23 12:10:39.214574 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.214546    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-16.ec2.internal" podStartSLOduration=21.214536812 podStartE2EDuration="21.214536812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:21.170907674 +0000 UTC m=+4.826416477" watchObservedRunningTime="2026-07-23 12:10:39.214536812 +0000 UTC m=+22.870045618"
Jul 23 12:10:39.234285 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.234240    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-m25tq" podStartSLOduration=2.976883151 podStartE2EDuration="22.234224499s" totalImagesPullingTime="19.257341348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:39.233927686 +0000 UTC m=+22.889436492" watchObservedRunningTime="2026-07-23 12:10:39.234224499 +0000 UTC m=+22.889733303"
Jul 23 12:10:39.261537 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.261516    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:39.293438 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.293394    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-89d2f" podStartSLOduration=3.262350938 podStartE2EDuration="22.293377894s" totalImagesPullingTime="19.031026956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:39.276227047 +0000 UTC m=+22.931735850" watchObservedRunningTime="2026-07-23 12:10:39.293377894 +0000 UTC m=+22.948886713"
Jul 23 12:10:39.293685 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.293665    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7wjgn" podStartSLOduration=3.228932527 podStartE2EDuration="22.293657667s" totalImagesPullingTime="19.06472514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:39.293224944 +0000 UTC m=+22.948733878" watchObservedRunningTime="2026-07-23 12:10:39.293657667 +0000 UTC m=+22.949166472"
Jul 23 12:10:39.309517 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.309478    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-16.ec2.internal" podStartSLOduration=21.309462672 podStartE2EDuration="21.309462672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:39.309352793 +0000 UTC m=+22.964861610" watchObservedRunningTime="2026-07-23 12:10:39.309462672 +0000 UTC m=+22.964971476"
Jul 23 12:10:39.328655 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:39.328612    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ncfx5" podStartSLOduration=3.272162561 podStartE2EDuration="22.328596817s" totalImagesPullingTime="19.056434256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:39.32834456 +0000 UTC m=+22.983853365" watchObservedRunningTime="2026-07-23 12:10:39.328596817 +0000 UTC m=+22.984105621"
Jul 23 12:10:40.044276 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:40.044246    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:40.044440 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:40.044353    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:40.044440 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:40.044417    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:40.044532 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:40.044509    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:40.212673 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:40.212642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vwrkm" event={"ID":"f6277871-5e9e-4f3b-bd37-248738e0c101","Type":"ContainerStarted","Data":"5b20c2d12f3371f254e5b45b1bf19221113842b120b1555e617bd8bb69107526"}
Jul 23 12:10:40.216775 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:40.216730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" event={"ID":"e6fa77ef-d668-4a5a-a79a-46bd992b1029","Type":"ContainerStarted","Data":"ce6345eed4936ce14352ab91e4c27f2d81dd697fb09823564e20b8e659840406"}
Jul 23 12:10:40.230622 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:40.230574    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vwrkm" podStartSLOduration=4.173390223 podStartE2EDuration="23.230555248s" totalImagesPullingTime="19.057165025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:40.230318504 +0000 UTC m=+23.885827308" watchObservedRunningTime="2026-07-23 12:10:40.230555248 +0000 UTC m=+23.886064051"
Jul 23 12:10:40.269653 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:40.269628    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 12:10:40.933907 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:40.933056    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T12:10:40.269650554Z","UUID":"4972f60e-4e7c-486b-aee1-9cead654fb62","Handler":null,"Name":"","Endpoint":""}
Jul 23 12:10:40.935406 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:40.935377    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 12:10:40.935549 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:40.935427    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 12:10:41.044456 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:41.044421    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:41.044622 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:41.044541    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:41.221099 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:41.221017    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26" event={"ID":"d387660f-5d09-497c-8aa3-16397dd07da7","Type":"ContainerStarted","Data":"00b841353d5d259209a83d4c266f89874c777aaf1458487b295eb3bab74b174b"}
Jul 23 12:10:42.044573 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:42.044542    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:42.044834 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:42.044654    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:42.044834 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:42.044714    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:42.044920 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:42.044868    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:42.226146 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:42.226109    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26" event={"ID":"d387660f-5d09-497c-8aa3-16397dd07da7","Type":"ContainerStarted","Data":"33d6bd37e43b9f182151b226e85ee8fa0c8abfd1d1a8966fe6e81c9454d6f95c"}
Jul 23 12:10:42.229705 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:42.229680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" event={"ID":"e6fa77ef-d668-4a5a-a79a-46bd992b1029","Type":"ContainerStarted","Data":"d2d65d3cc63fb8fb088962220e3617ca7a0e0d7cfd8a92f4643c11cd4726b2bd"}
Jul 23 12:10:42.246353 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:42.246308    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sjh26" podStartSLOduration=3.718767721 podStartE2EDuration="25.246290677s" totalImagesPullingTime="21.527522956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:42.24592288 +0000 UTC m=+25.901431686" watchObservedRunningTime="2026-07-23 12:10:42.246290677 +0000 UTC m=+25.901799484"
Jul 23 12:10:43.047363 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:43.047187    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:43.047531 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:43.047434    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:43.700557 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:43.700533    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:43.701696 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:43.701681    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:44.044660 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.044468    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:44.044843 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.044540    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:44.044843 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:44.044736    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:44.044949 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:44.044864    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:44.238803 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.238747    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" event={"ID":"e6fa77ef-d668-4a5a-a79a-46bd992b1029","Type":"ContainerStarted","Data":"a1d7e3ccd5c230f41598c25c865c70155ca77a20c4bfb7cd88e33038e02b8b8f"}
Jul 23 12:10:44.239054 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.239029    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:44.239142 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.239099    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:44.239207 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.239143    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:44.240806 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.240779    2565 generic.go:356] "Generic (PLEG): container finished" podID="17efe563-dcf8-4bf3-9c27-dbdecf122831" containerID="503a1e31e1e9bafee6630694a9e30e72a133da05c42bccd6b8f1e5383d900627" exitCode=0
Jul 23 12:10:44.240930 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.240871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrgwq" event={"ID":"17efe563-dcf8-4bf3-9c27-dbdecf122831","Type":"ContainerDied","Data":"503a1e31e1e9bafee6630694a9e30e72a133da05c42bccd6b8f1e5383d900627"}
Jul 23 12:10:44.241602 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.241551    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7wjgn"
Jul 23 12:10:44.258124 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.258100    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:44.269423 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.269378    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl" podStartSLOduration=8.171650696 podStartE2EDuration="27.269362851s" totalImagesPullingTime="19.097712155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:44.268732798 +0000 UTC m=+27.924241601" watchObservedRunningTime="2026-07-23 12:10:44.269362851 +0000 UTC m=+27.924871654"
Jul 23 12:10:44.279068 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:44.279047    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:10:45.043620 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:45.043579    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:45.044151 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:45.043685    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:45.664408 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:45.664360    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f22wm"]
Jul 23 12:10:45.664676 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:45.664486    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:45.664676 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:45.664591    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:45.666959 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:45.666933    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tkw2s"]
Jul 23 12:10:45.667067 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:45.667038    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:45.667155 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:45.667133    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:45.679154 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:45.679057    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xb7r6"]
Jul 23 12:10:45.679274 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:45.679168    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:45.679274 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:45.679253    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:46.247308 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:46.247271    2565 generic.go:356] "Generic (PLEG): container finished" podID="17efe563-dcf8-4bf3-9c27-dbdecf122831" containerID="eb0a8f7e22802e2294e16b575b855e635cdaa044130564d2b7017ce844baf42b" exitCode=0
Jul 23 12:10:46.247806 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:46.247365    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrgwq" event={"ID":"17efe563-dcf8-4bf3-9c27-dbdecf122831","Type":"ContainerDied","Data":"eb0a8f7e22802e2294e16b575b855e635cdaa044130564d2b7017ce844baf42b"}
Jul 23 12:10:47.044857 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:47.044827    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:47.045030 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:47.044950    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:48.044737 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:48.044490    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:48.045211 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:48.044521    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:48.045211 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:48.044797    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:48.045211 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:48.044886    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:48.253577 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:48.253547    2565 generic.go:356] "Generic (PLEG): container finished" podID="17efe563-dcf8-4bf3-9c27-dbdecf122831" containerID="fa1f0b6e0d83cdfe485871cf64c17038d8b0b6200d15d8032ae576255df66508" exitCode=0
Jul 23 12:10:48.253725 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:48.253610    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrgwq" event={"ID":"17efe563-dcf8-4bf3-9c27-dbdecf122831","Type":"ContainerDied","Data":"fa1f0b6e0d83cdfe485871cf64c17038d8b0b6200d15d8032ae576255df66508"}
Jul 23 12:10:49.044333 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:49.044300    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:49.044560 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:49.044442    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:50.043616 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:50.043537    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:50.044104 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.043669    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xb7r6" podUID="ccd0a997-8423-410e-bbda-4cc49e6c94d9"
Jul 23 12:10:50.044104 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:50.043726    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:50.044104 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.043856    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tkw2s" podUID="501c1bcd-51e9-4cc3-9356-ab8ed8de7a87"
Jul 23 12:10:50.625897 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:50.625865    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:50.625897 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:50.625910    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:50.626146 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.626036    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:50.626146 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.626044    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:50.626146 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.626095    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret podName:aa53ff2f-84ce-4767-8e00-a0230504134b nodeName:}" failed. No retries permitted until 2026-07-23 12:11:22.626077786 +0000 UTC m=+66.281586569 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret") pod "global-pull-secret-syncer-f22wm" (UID: "aa53ff2f-84ce-4767-8e00-a0230504134b") : object "kube-system"/"original-pull-secret" not registered
Jul 23 12:10:50.626146 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.626114    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs podName:ccd0a997-8423-410e-bbda-4cc49e6c94d9 nodeName:}" failed. No retries permitted until 2026-07-23 12:11:22.626104922 +0000 UTC m=+66.281613708 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs") pod "network-metrics-daemon-xb7r6" (UID: "ccd0a997-8423-410e-bbda-4cc49e6c94d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 12:10:50.727268 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:50.726780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:50.727268 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.726954    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 12:10:50.727511 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.727311    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 12:10:50.727511 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.727329    2565 projected.go:196] Error preparing data for projected volume kube-api-access-t7vgf for pod openshift-network-diagnostics/network-check-target-tkw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:50.727511 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:50.727410    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf podName:501c1bcd-51e9-4cc3-9356-ab8ed8de7a87 nodeName:}" failed. No retries permitted until 2026-07-23 12:11:22.727389803 +0000 UTC m=+66.382898598 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t7vgf" (UniqueName: "kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf") pod "network-check-target-tkw2s" (UID: "501c1bcd-51e9-4cc3-9356-ab8ed8de7a87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 12:10:50.807773 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:50.807740    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-89d2f_c24d092b-c557-43b9-a1ce-dd4d7eb8f55c/dns-node-resolver/0.log"
Jul 23 12:10:51.044317 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.044239    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:51.044788 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:51.044387    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f22wm" podUID="aa53ff2f-84ce-4767-8e00-a0230504134b"
Jul 23 12:10:51.587858 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.587820    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-16.ec2.internal" event="NodeReady"
Jul 23 12:10:51.588043 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.588008    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 12:10:51.640474 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.640447    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2xtcb"]
Jul 23 12:10:51.643297 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.643266    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2xtcb"
Jul 23 12:10:51.647161 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.647137    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 12:10:51.647324 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.647235    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 12:10:51.647324 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.647241    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2gntm\""
Jul 23 12:10:51.647442 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.647336    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 12:10:51.648270 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.648246    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vsx98"]
Jul 23 12:10:51.650365 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.650327    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.653053 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.652881    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2xtcb"]
Jul 23 12:10:51.653837 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.653814    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 12:10:51.653837 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.653831    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l7bff\""
Jul 23 12:10:51.653993 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.653892    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 12:10:51.653993 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.653925    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 12:10:51.653993 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.653981    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 12:10:51.661011 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.660947    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vsx98"]
Jul 23 12:10:51.664735 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.664714    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xzfkw"]
Jul 23 12:10:51.666894 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.666874    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.669749 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.669723    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 12:10:51.669749 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.669745    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 12:10:51.669915 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.669777    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dbmrb\""
Jul 23 12:10:51.683532 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.683504    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xzfkw"]
Jul 23 12:10:51.734775 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.734722    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/008cf23c-b927-426d-9a6a-e69d58a5ba3b-data-volume\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.734971 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.734783    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.734971 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.734808    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hndq8\" (UniqueName: \"kubernetes.io/projected/17da2583-078d-4eb3-88f2-640d28af40ba-kube-api-access-hndq8\") pod \"ingress-canary-2xtcb\" (UID: \"17da2583-078d-4eb3-88f2-640d28af40ba\") " pod="openshift-ingress-canary/ingress-canary-2xtcb"
Jul 23 12:10:51.734971 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.734906    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/008cf23c-b927-426d-9a6a-e69d58a5ba3b-crio-socket\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.734971 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.734934    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72vrj\" (UniqueName: \"kubernetes.io/projected/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-api-access-72vrj\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.734971 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.734963    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17da2583-078d-4eb3-88f2-640d28af40ba-cert\") pod \"ingress-canary-2xtcb\" (UID: \"17da2583-078d-4eb3-88f2-640d28af40ba\") " pod="openshift-ingress-canary/ingress-canary-2xtcb"
Jul 23 12:10:51.735224 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.734993    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.772647 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.772616    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9dtwz_52ac0e67-2df0-4d14-84b1-06ba64808ccf/node-ca/0.log"
Jul 23 12:10:51.835703 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835670    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-config-volume\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.835883 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkmbb\" (UniqueName: \"kubernetes.io/projected/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-kube-api-access-xkmbb\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.835883 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835785    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/008cf23c-b927-426d-9a6a-e69d58a5ba3b-data-volume\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.835883 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835804    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.835883 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835827    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hndq8\" (UniqueName: \"kubernetes.io/projected/17da2583-078d-4eb3-88f2-640d28af40ba-kube-api-access-hndq8\") pod \"ingress-canary-2xtcb\" (UID: \"17da2583-078d-4eb3-88f2-640d28af40ba\") " pod="openshift-ingress-canary/ingress-canary-2xtcb"
Jul 23 12:10:51.835883 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/008cf23c-b927-426d-9a6a-e69d58a5ba3b-crio-socket\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.835883 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835879    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72vrj\" (UniqueName: \"kubernetes.io/projected/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-api-access-72vrj\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.836180 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835902    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17da2583-078d-4eb3-88f2-640d28af40ba-cert\") pod \"ingress-canary-2xtcb\" (UID: \"17da2583-078d-4eb3-88f2-640d28af40ba\") " pod="openshift-ingress-canary/ingress-canary-2xtcb"
Jul 23 12:10:51.836180 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835918    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.836180 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835934    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-tmp-dir\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.836180 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.835952    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.836180 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:51.835959    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:51.836180 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:51.836023    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:52.336003083 +0000 UTC m=+35.991511868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:51.836180 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:51.836038    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 12:10:51.836180 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:51.836095    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:52.336081868 +0000 UTC m=+35.991590667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : configmap "kube-rbac-proxy" not found
Jul 23 12:10:51.836180 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.836170    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/008cf23c-b927-426d-9a6a-e69d58a5ba3b-data-volume\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.836584 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.836278    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/008cf23c-b927-426d-9a6a-e69d58a5ba3b-crio-socket\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.840734 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.840665    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17da2583-078d-4eb3-88f2-640d28af40ba-cert\") pod \"ingress-canary-2xtcb\" (UID: \"17da2583-078d-4eb3-88f2-640d28af40ba\") " pod="openshift-ingress-canary/ingress-canary-2xtcb"
Jul 23 12:10:51.846465 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.846441    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hndq8\" (UniqueName: \"kubernetes.io/projected/17da2583-078d-4eb3-88f2-640d28af40ba-kube-api-access-hndq8\") pod \"ingress-canary-2xtcb\" (UID: \"17da2583-078d-4eb3-88f2-640d28af40ba\") " pod="openshift-ingress-canary/ingress-canary-2xtcb"
Jul 23 12:10:51.846704 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.846685    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72vrj\" (UniqueName: \"kubernetes.io/projected/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-api-access-72vrj\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:51.936650 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.936620    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.936650 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.936647    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-tmp-dir\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.936923 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.936688    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-config-volume\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.936923 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:51.936806    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 12:10:51.936923 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.936824    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkmbb\" (UniqueName: \"kubernetes.io/projected/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-kube-api-access-xkmbb\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.936923 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:51.936885    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls podName:4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:52.436866971 +0000 UTC m=+36.092375765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls") pod "dns-default-xzfkw" (UID: "4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b") : secret "dns-default-metrics-tls" not found
Jul 23 12:10:51.937115 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.937043    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-tmp-dir\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.937340 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.937320    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-config-volume\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.946701 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.946676    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkmbb\" (UniqueName: \"kubernetes.io/projected/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-kube-api-access-xkmbb\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:51.956661 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:51.956638    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2xtcb"
Jul 23 12:10:52.043800 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.043754    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:10:52.043800 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.043810    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:10:52.050976 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.050949    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5z96\""
Jul 23 12:10:52.051726 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.050992    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pbg55\""
Jul 23 12:10:52.051726 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.051446    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 12:10:52.051726 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.051492    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 12:10:52.051726 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.051697    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 12:10:52.341228 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.341189    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:52.341434 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.341253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:52.341434 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:52.341348    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 12:10:52.341434 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:52.341355    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:52.341434 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:52.341409    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:53.34139203 +0000 UTC m=+36.996900826 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : configmap "kube-rbac-proxy" not found
Jul 23 12:10:52.341434 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:52.341427    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:53.34141854 +0000 UTC m=+36.996927322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:52.442669 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.442627    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:52.442882 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:52.442795    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 12:10:52.442882 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:52.442873    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls podName:4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:53.442850588 +0000 UTC m=+37.098359372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls") pod "dns-default-xzfkw" (UID: "4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b") : secret "dns-default-metrics-tls" not found
Jul 23 12:10:52.619151 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:52.618989    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2xtcb"]
Jul 23 12:10:52.622434 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:52.622411    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17da2583_078d_4eb3_88f2_640d28af40ba.slice/crio-b5735cc8eb2aa2250266b2b30bf46888ca2519d63c1f395b45f32d4b8235b51d WatchSource:0}: Error finding container b5735cc8eb2aa2250266b2b30bf46888ca2519d63c1f395b45f32d4b8235b51d: Status 404 returned error can't find the container with id b5735cc8eb2aa2250266b2b30bf46888ca2519d63c1f395b45f32d4b8235b51d
Jul 23 12:10:53.044267 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:53.044190    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:10:53.047514 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:53.047493    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 12:10:53.269450 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:53.269412    2565 generic.go:356] "Generic (PLEG): container finished" podID="17efe563-dcf8-4bf3-9c27-dbdecf122831" containerID="f9201fd7ae57a7afe77b75a8de2933fd041329c66da6f10d09eb94e07e8e63ac" exitCode=0
Jul 23 12:10:53.269881 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:53.269506    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrgwq" event={"ID":"17efe563-dcf8-4bf3-9c27-dbdecf122831","Type":"ContainerDied","Data":"f9201fd7ae57a7afe77b75a8de2933fd041329c66da6f10d09eb94e07e8e63ac"}
Jul 23 12:10:53.271199 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:53.271172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2xtcb" event={"ID":"17da2583-078d-4eb3-88f2-640d28af40ba","Type":"ContainerStarted","Data":"b5735cc8eb2aa2250266b2b30bf46888ca2519d63c1f395b45f32d4b8235b51d"}
Jul 23 12:10:53.349001 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:53.348970    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:53.349168 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:53.349040    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:53.349168 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:53.349137    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 12:10:53.349168 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:53.349149    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:53.349330 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:53.349202    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:55.349184188 +0000 UTC m=+39.004692969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : configmap "kube-rbac-proxy" not found
Jul 23 12:10:53.349330 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:53.349221    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:55.349211196 +0000 UTC m=+39.004719984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:53.450395 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:53.450356    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:53.450699 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:53.450666    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 12:10:53.450834 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:53.450749    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls podName:4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:55.45073116 +0000 UTC m=+39.106239957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls") pod "dns-default-xzfkw" (UID: "4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b") : secret "dns-default-metrics-tls" not found
Jul 23 12:10:54.276319 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:54.276287    2565 generic.go:356] "Generic (PLEG): container finished" podID="17efe563-dcf8-4bf3-9c27-dbdecf122831" containerID="a332f1eabaf8bac0fba882870c2bc5b78ce053681f9d402aaa780a0a267e53d6" exitCode=0
Jul 23 12:10:54.276664 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:54.276351    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrgwq" event={"ID":"17efe563-dcf8-4bf3-9c27-dbdecf122831","Type":"ContainerDied","Data":"a332f1eabaf8bac0fba882870c2bc5b78ce053681f9d402aaa780a0a267e53d6"}
Jul 23 12:10:55.282903 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:55.282864    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrgwq" event={"ID":"17efe563-dcf8-4bf3-9c27-dbdecf122831","Type":"ContainerStarted","Data":"d0b6dbdf7ca22200c79109005b4e2d06beb4ed5cbd8f8b5349df627776a6bd28"}
Jul 23 12:10:55.313791 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:55.313719    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mrgwq" podStartSLOduration=7.879596914 podStartE2EDuration="38.313699824s" totalImagesPullingTime="30.43410291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:55.313190501 +0000 UTC m=+38.968699321" watchObservedRunningTime="2026-07-23 12:10:55.313699824 +0000 UTC m=+38.969208624"
Jul 23 12:10:55.363969 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:55.363937    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:55.364160 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:55.363993    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:55.364160 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:55.364083    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 12:10:55.364160 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:55.364112    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:55.364160 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:55.364150    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:59.364131719 +0000 UTC m=+43.019640504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : configmap "kube-rbac-proxy" not found
Jul 23 12:10:55.364420 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:55.364178    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:59.364161011 +0000 UTC m=+43.019669795 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:55.464846 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:55.464746    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:55.464983 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:55.464865    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 12:10:55.464983 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:55.464924    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls podName:4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b nodeName:}" failed. No retries permitted until 2026-07-23 12:10:59.464909583 +0000 UTC m=+43.120418387 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls") pod "dns-default-xzfkw" (UID: "4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b") : secret "dns-default-metrics-tls" not found
Jul 23 12:10:56.286435 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:56.286350    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2xtcb" event={"ID":"17da2583-078d-4eb3-88f2-640d28af40ba","Type":"ContainerStarted","Data":"16f5bb1082414874c4446a03f6487dba5486298ab06c29522436d4f90e67cade"}
Jul 23 12:10:56.312429 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:56.312377    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2xtcb" podStartSLOduration=2.127809403 podStartE2EDuration="5.312357236s" totalImagesPullingTime="3.184547833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:56.312077098 +0000 UTC m=+39.967585902" watchObservedRunningTime="2026-07-23 12:10:56.312357236 +0000 UTC m=+39.967866238"
Jul 23 12:10:59.394488 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:59.394448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:59.394985 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:59.394558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:10:59.394985 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:59.394561    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 12:10:59.394985 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:59.394674    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:11:07.394654164 +0000 UTC m=+51.050163173 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : configmap "kube-rbac-proxy" not found
Jul 23 12:10:59.394985 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:59.394705    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:59.394985 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:10:59.394781    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls podName:008cf23c-b927-426d-9a6a-e69d58a5ba3b nodeName:}" failed. No retries permitted until 2026-07-23 12:11:07.394748228 +0000 UTC m=+51.050257011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vsx98" (UID: "008cf23c-b927-426d-9a6a-e69d58a5ba3b") : secret "insights-runtime-extractor-tls" not found
Jul 23 12:10:59.494984 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:59.494943    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:59.498933 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:59.498907    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b-metrics-tls\") pod \"dns-default-xzfkw\" (UID: \"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b\") " pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:59.776881 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:59.776791    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:10:59.932233 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:10:59.932200    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xzfkw"]
Jul 23 12:10:59.935376 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:10:59.935347    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e49b1de_e7f2_41f8_b9d1_e2a8a21e360b.slice/crio-ea4c3618abef1bba8588ba4a7d5188361023487d998814737fd4f50c3e2c0d1c WatchSource:0}: Error finding container ea4c3618abef1bba8588ba4a7d5188361023487d998814737fd4f50c3e2c0d1c: Status 404 returned error can't find the container with id ea4c3618abef1bba8588ba4a7d5188361023487d998814737fd4f50c3e2c0d1c
Jul 23 12:11:00.297047 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:00.297015    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xzfkw" event={"ID":"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b","Type":"ContainerStarted","Data":"ea4c3618abef1bba8588ba4a7d5188361023487d998814737fd4f50c3e2c0d1c"}
Jul 23 12:11:03.306038 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:03.305999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xzfkw" event={"ID":"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b","Type":"ContainerStarted","Data":"dff41ce18da4e189cad2f9bcb75ec568ca7e109c22899721190188b689183490"}
Jul 23 12:11:03.306451 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:03.306062    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xzfkw" event={"ID":"4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b","Type":"ContainerStarted","Data":"42fa5082b637619517a82506bdae6019cdbbb38b726f7d2cfe4df0e3b4b06801"}
Jul 23 12:11:03.306451 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:03.306105    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:11:03.326717 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:03.326660    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xzfkw" podStartSLOduration=9.880304289 podStartE2EDuration="12.326637887s" totalImagesPullingTime="2.446333598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:03.326200135 +0000 UTC m=+46.981708939" watchObservedRunningTime="2026-07-23 12:11:03.326637887 +0000 UTC m=+46.982146686"
Jul 23 12:11:07.448014 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:07.447973    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:11:07.448397 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:07.448152    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:11:07.448483 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:07.448463    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/008cf23c-b927-426d-9a6a-e69d58a5ba3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:11:07.450336 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:07.450316    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/008cf23c-b927-426d-9a6a-e69d58a5ba3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vsx98\" (UID: \"008cf23c-b927-426d-9a6a-e69d58a5ba3b\") " pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:11:07.564798 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:07.564752    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vsx98"
Jul 23 12:11:07.710755 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:07.710670    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vsx98"]
Jul 23 12:11:07.714944 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:11:07.714910    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod008cf23c_b927_426d_9a6a_e69d58a5ba3b.slice/crio-682b20f2a7215435e22e441c4537d3220ecb73668b8b991903c620bd862b4a96 WatchSource:0}: Error finding container 682b20f2a7215435e22e441c4537d3220ecb73668b8b991903c620bd862b4a96: Status 404 returned error can't find the container with id 682b20f2a7215435e22e441c4537d3220ecb73668b8b991903c620bd862b4a96
Jul 23 12:11:08.319288 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:08.319209    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vsx98" event={"ID":"008cf23c-b927-426d-9a6a-e69d58a5ba3b","Type":"ContainerStarted","Data":"682b20f2a7215435e22e441c4537d3220ecb73668b8b991903c620bd862b4a96"}
Jul 23 12:11:12.481972 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.481933    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"]
Jul 23 12:11:12.488512 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.488486    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"]
Jul 23 12:11:12.488655 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.488602    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"
Jul 23 12:11:12.491188 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.491159    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 23 12:11:12.491662 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.491635    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 23 12:11:12.491846 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.491823    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"]
Jul 23 12:11:12.491942 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.491880    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-79k7s\""
Jul 23 12:11:12.492014 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.491998    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"
Jul 23 12:11:12.494280 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.494260    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"]
Jul 23 12:11:12.494389 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.494346    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.495031 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.495008    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4rgtr\""
Jul 23 12:11:12.495409 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.495250    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 23 12:11:12.495950 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.495928    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 23 12:11:12.495950 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.495944    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 23 12:11:12.496150 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.495994    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 23 12:11:12.496795 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.496777    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 23 12:11:12.497827 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.497809    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"]
Jul 23 12:11:12.497920 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.497840    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"]
Jul 23 12:11:12.497981 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.497930    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.500301 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.500285    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"]
Jul 23 12:11:12.502349 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.501923    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 23 12:11:12.502443 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.501968    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 23 12:11:12.502495 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.502021    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 23 12:11:12.502572 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.502019    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 23 12:11:12.502973 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.502953    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"]
Jul 23 12:11:12.588738 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.588708    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3848ca6a-ae6a-470d-ad4f-b1cadc0fb629-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-h2w84\" (UID: \"3848ca6a-ae6a-470d-ad4f-b1cadc0fb629\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"
Jul 23 12:11:12.588918 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.588781    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.588918 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.588844    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3848ca6a-ae6a-470d-ad4f-b1cadc0fb629-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-h2w84\" (UID: \"3848ca6a-ae6a-470d-ad4f-b1cadc0fb629\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"
Jul 23 12:11:12.588918 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.588883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5150f4ce-a6ad-43e6-8a42-c2d3b220699d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b56f59fb-r9z4c\" (UID: \"5150f4ce-a6ad-43e6-8a42-c2d3b220699d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"
Jul 23 12:11:12.588918 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.588911    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rw6s\" (UniqueName: \"kubernetes.io/projected/5320ec8a-cef3-4d70-89b2-6408bd503bda-kube-api-access-5rw6s\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.589126 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.588943    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/269e6fa5-8d9f-4359-92dc-4980111e8713-tmp\") pod \"klusterlet-addon-workmgr-7d74b4845f-6r2v7\" (UID: \"269e6fa5-8d9f-4359-92dc-4980111e8713\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.589126 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.588981    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.589126 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.589050    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-ca\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.589126 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.589075    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5320ec8a-cef3-4d70-89b2-6408bd503bda-additional-service-ca\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.589126 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.589096    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-hub\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.589126 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.589125    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn7fm\" (UniqueName: \"kubernetes.io/projected/5150f4ce-a6ad-43e6-8a42-c2d3b220699d-kube-api-access-mn7fm\") pod \"managed-serviceaccount-addon-agent-5b56f59fb-r9z4c\" (UID: \"5150f4ce-a6ad-43e6-8a42-c2d3b220699d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"
Jul 23 12:11:12.589348 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.589170    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t7qz\" (UniqueName: \"kubernetes.io/projected/269e6fa5-8d9f-4359-92dc-4980111e8713-kube-api-access-5t7qz\") pod \"klusterlet-addon-workmgr-7d74b4845f-6r2v7\" (UID: \"269e6fa5-8d9f-4359-92dc-4980111e8713\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.589348 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.589211    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/269e6fa5-8d9f-4359-92dc-4980111e8713-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d74b4845f-6r2v7\" (UID: \"269e6fa5-8d9f-4359-92dc-4980111e8713\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.690401 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690369    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3848ca6a-ae6a-470d-ad4f-b1cadc0fb629-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-h2w84\" (UID: \"3848ca6a-ae6a-470d-ad4f-b1cadc0fb629\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"
Jul 23 12:11:12.690586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.690586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690453    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3848ca6a-ae6a-470d-ad4f-b1cadc0fb629-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-h2w84\" (UID: \"3848ca6a-ae6a-470d-ad4f-b1cadc0fb629\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"
Jul 23 12:11:12.690586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690484    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5150f4ce-a6ad-43e6-8a42-c2d3b220699d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b56f59fb-r9z4c\" (UID: \"5150f4ce-a6ad-43e6-8a42-c2d3b220699d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"
Jul 23 12:11:12.690586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rw6s\" (UniqueName: \"kubernetes.io/projected/5320ec8a-cef3-4d70-89b2-6408bd503bda-kube-api-access-5rw6s\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.690586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690524    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/269e6fa5-8d9f-4359-92dc-4980111e8713-tmp\") pod \"klusterlet-addon-workmgr-7d74b4845f-6r2v7\" (UID: \"269e6fa5-8d9f-4359-92dc-4980111e8713\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.690586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690542    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.690586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690565    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-ca\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.690586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690588    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5320ec8a-cef3-4d70-89b2-6408bd503bda-additional-service-ca\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.691015 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690608    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-hub\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.691015 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690630    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn7fm\" (UniqueName: \"kubernetes.io/projected/5150f4ce-a6ad-43e6-8a42-c2d3b220699d-kube-api-access-mn7fm\") pod \"managed-serviceaccount-addon-agent-5b56f59fb-r9z4c\" (UID: \"5150f4ce-a6ad-43e6-8a42-c2d3b220699d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"
Jul 23 12:11:12.691015 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690658    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t7qz\" (UniqueName: \"kubernetes.io/projected/269e6fa5-8d9f-4359-92dc-4980111e8713-kube-api-access-5t7qz\") pod \"klusterlet-addon-workmgr-7d74b4845f-6r2v7\" (UID: \"269e6fa5-8d9f-4359-92dc-4980111e8713\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.691015 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.690688    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/269e6fa5-8d9f-4359-92dc-4980111e8713-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d74b4845f-6r2v7\" (UID: \"269e6fa5-8d9f-4359-92dc-4980111e8713\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.691204 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.691155    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3848ca6a-ae6a-470d-ad4f-b1cadc0fb629-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-h2w84\" (UID: \"3848ca6a-ae6a-470d-ad4f-b1cadc0fb629\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"
Jul 23 12:11:12.692113 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.692041    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5320ec8a-cef3-4d70-89b2-6408bd503bda-additional-service-ca\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.692234 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.692128    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/269e6fa5-8d9f-4359-92dc-4980111e8713-tmp\") pod \"klusterlet-addon-workmgr-7d74b4845f-6r2v7\" (UID: \"269e6fa5-8d9f-4359-92dc-4980111e8713\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.693990 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.693943    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5150f4ce-a6ad-43e6-8a42-c2d3b220699d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b56f59fb-r9z4c\" (UID: \"5150f4ce-a6ad-43e6-8a42-c2d3b220699d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"
Jul 23 12:11:12.694158 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.694115    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3848ca6a-ae6a-470d-ad4f-b1cadc0fb629-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-h2w84\" (UID: \"3848ca6a-ae6a-470d-ad4f-b1cadc0fb629\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"
Jul 23 12:11:12.694158 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.694127    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/269e6fa5-8d9f-4359-92dc-4980111e8713-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d74b4845f-6r2v7\" (UID: \"269e6fa5-8d9f-4359-92dc-4980111e8713\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.694380 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.694355    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-hub\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.694626 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.694608    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-ca\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.694692 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.694611    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.694911 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.694890    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5320ec8a-cef3-4d70-89b2-6408bd503bda-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.701593 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.701566    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rw6s\" (UniqueName: \"kubernetes.io/projected/5320ec8a-cef3-4d70-89b2-6408bd503bda-kube-api-access-5rw6s\") pod \"cluster-proxy-proxy-agent-796f67c447-glg5d\" (UID: \"5320ec8a-cef3-4d70-89b2-6408bd503bda\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:12.701713 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.701658    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t7qz\" (UniqueName: \"kubernetes.io/projected/269e6fa5-8d9f-4359-92dc-4980111e8713-kube-api-access-5t7qz\") pod \"klusterlet-addon-workmgr-7d74b4845f-6r2v7\" (UID: \"269e6fa5-8d9f-4359-92dc-4980111e8713\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.701974 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.701940    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn7fm\" (UniqueName: \"kubernetes.io/projected/5150f4ce-a6ad-43e6-8a42-c2d3b220699d-kube-api-access-mn7fm\") pod \"managed-serviceaccount-addon-agent-5b56f59fb-r9z4c\" (UID: \"5150f4ce-a6ad-43e6-8a42-c2d3b220699d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"
Jul 23 12:11:12.803319 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.803244    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"
Jul 23 12:11:12.824247 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.824209    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"
Jul 23 12:11:12.833195 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.833164    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:12.838247 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:12.838221    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"
Jul 23 12:11:13.029905 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.029779    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"]
Jul 23 12:11:13.032699 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:11:13.032665    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269e6fa5_8d9f_4359_92dc_4980111e8713.slice/crio-2f14209ba257920d9ded2019143fb9a7c0b1bec6277af3d763e84b4efbd93300 WatchSource:0}: Error finding container 2f14209ba257920d9ded2019143fb9a7c0b1bec6277af3d763e84b4efbd93300: Status 404 returned error can't find the container with id 2f14209ba257920d9ded2019143fb9a7c0b1bec6277af3d763e84b4efbd93300
Jul 23 12:11:13.257238 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.257201    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84"]
Jul 23 12:11:13.261635 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.261605    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d"]
Jul 23 12:11:13.261934 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:11:13.261898    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3848ca6a_ae6a_470d_ad4f_b1cadc0fb629.slice/crio-83ed99e3409aa2aa66b5228e4c5961a9110cf7bd8884e18dafa0a498a85f3768 WatchSource:0}: Error finding container 83ed99e3409aa2aa66b5228e4c5961a9110cf7bd8884e18dafa0a498a85f3768: Status 404 returned error can't find the container with id 83ed99e3409aa2aa66b5228e4c5961a9110cf7bd8884e18dafa0a498a85f3768
Jul 23 12:11:13.264638 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.264611    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c"]
Jul 23 12:11:13.265699 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:11:13.265679    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5320ec8a_cef3_4d70_89b2_6408bd503bda.slice/crio-72c7bceaab9a7c07c90624b9c61809fb4975154e8abbdaefaa0c379a176c6597 WatchSource:0}: Error finding container 72c7bceaab9a7c07c90624b9c61809fb4975154e8abbdaefaa0c379a176c6597: Status 404 returned error can't find the container with id 72c7bceaab9a7c07c90624b9c61809fb4975154e8abbdaefaa0c379a176c6597
Jul 23 12:11:13.268159 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:11:13.268142    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5150f4ce_a6ad_43e6_8a42_c2d3b220699d.slice/crio-8533ad3bf1b73cf02541ecf6f8da0913974e69acc440c6a8c68dce8358f91d84 WatchSource:0}: Error finding container 8533ad3bf1b73cf02541ecf6f8da0913974e69acc440c6a8c68dce8358f91d84: Status 404 returned error can't find the container with id 8533ad3bf1b73cf02541ecf6f8da0913974e69acc440c6a8c68dce8358f91d84
Jul 23 12:11:13.310716 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.310692    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xzfkw"
Jul 23 12:11:13.334023 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.333985    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84" event={"ID":"3848ca6a-ae6a-470d-ad4f-b1cadc0fb629","Type":"ContainerStarted","Data":"83ed99e3409aa2aa66b5228e4c5961a9110cf7bd8884e18dafa0a498a85f3768"}
Jul 23 12:11:13.335161 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.335142    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7" event={"ID":"269e6fa5-8d9f-4359-92dc-4980111e8713","Type":"ContainerStarted","Data":"2f14209ba257920d9ded2019143fb9a7c0b1bec6277af3d763e84b4efbd93300"}
Jul 23 12:11:13.336533 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.336505    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vsx98" event={"ID":"008cf23c-b927-426d-9a6a-e69d58a5ba3b","Type":"ContainerStarted","Data":"4ad518e72e41bf1c7f981ba4c9820a14fa024bfc02b339b303fdb531ea470c03"}
Jul 23 12:11:13.337538 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.337519    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c" event={"ID":"5150f4ce-a6ad-43e6-8a42-c2d3b220699d","Type":"ContainerStarted","Data":"8533ad3bf1b73cf02541ecf6f8da0913974e69acc440c6a8c68dce8358f91d84"}
Jul 23 12:11:13.338806 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:13.338780    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d" event={"ID":"5320ec8a-cef3-4d70-89b2-6408bd503bda","Type":"ContainerStarted","Data":"72c7bceaab9a7c07c90624b9c61809fb4975154e8abbdaefaa0c379a176c6597"}
Jul 23 12:11:16.264160 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:16.263692    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tpwhl"
Jul 23 12:11:21.377594 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.377547    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84" event={"ID":"3848ca6a-ae6a-470d-ad4f-b1cadc0fb629","Type":"ContainerStarted","Data":"6035a38b1fa2038c0da5c4fa6f092de7b0bd94328fbb9efbec4d1081c0ac8106"}
Jul 23 12:11:21.380150 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.380119    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7" event={"ID":"269e6fa5-8d9f-4359-92dc-4980111e8713","Type":"ContainerStarted","Data":"9fe8849ac108736810f1267e1732f87b4c4e9102d928aa6fa442dc53267d097f"}
Jul 23 12:11:21.380487 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.380471    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:21.381403 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.381364    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7" podUID="269e6fa5-8d9f-4359-92dc-4980111e8713" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.12:8000/readyz\": dial tcp 10.134.0.12:8000: connect: connection refused"
Jul 23 12:11:21.382727 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.382703    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vsx98" event={"ID":"008cf23c-b927-426d-9a6a-e69d58a5ba3b","Type":"ContainerStarted","Data":"ca1423b2116f2f87fb82d1b1ee7fe80870a7d8e09f7400b71a9f57d5290fdf15"}
Jul 23 12:11:21.384398 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.384356    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c" event={"ID":"5150f4ce-a6ad-43e6-8a42-c2d3b220699d","Type":"ContainerStarted","Data":"8af5d48b70065f1a16392637034ae711444cc3c57f450fffea41b7a021ccce9c"}
Jul 23 12:11:21.386260 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.386234    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d" event={"ID":"5320ec8a-cef3-4d70-89b2-6408bd503bda","Type":"ContainerStarted","Data":"b2a23847be086c0ada8e95f0d31c7817cd537bc71f68921025457344ec8109b0"}
Jul 23 12:11:21.400377 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.400264    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-h2w84" podStartSLOduration=1.513229515 podStartE2EDuration="9.400243418s" totalImagesPullingTime="7.887013903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:21.398652897 +0000 UTC m=+65.054161702" watchObservedRunningTime="2026-07-23 12:11:21.400243418 +0000 UTC m=+65.055752223"
Jul 23 12:11:21.421786 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.421711    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7" podStartSLOduration=1.20931074 podStartE2EDuration="9.421688901s" totalImagesPullingTime="8.212378161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:21.420705948 +0000 UTC m=+65.076214754" watchObservedRunningTime="2026-07-23 12:11:21.421688901 +0000 UTC m=+65.077197706"
Jul 23 12:11:21.444146 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:21.444092    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b56f59fb-r9z4c" podStartSLOduration=1.43985047 podStartE2EDuration="9.44407025s" totalImagesPullingTime="8.00421978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:21.443633301 +0000 UTC m=+65.099142105" watchObservedRunningTime="2026-07-23 12:11:21.44407025 +0000 UTC m=+65.099579055"
Jul 23 12:11:22.392716 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.392677    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d" event={"ID":"5320ec8a-cef3-4d70-89b2-6408bd503bda","Type":"ContainerStarted","Data":"0d63e3d931972c01b33065c4728b99b5e1abc6cfc6ff6b7e737a59e63cdc5c1d"}
Jul 23 12:11:22.393204 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.392729    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d" event={"ID":"5320ec8a-cef3-4d70-89b2-6408bd503bda","Type":"ContainerStarted","Data":"f5c2aa95331000aeb000d27641b8a727cf3b671d66950ccc149c976f5aac376b"}
Jul 23 12:11:22.394083 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.394064    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d74b4845f-6r2v7"
Jul 23 12:11:22.414561 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.414500    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-796f67c447-glg5d" podStartSLOduration=2.433908616 podStartE2EDuration="10.414481666s" totalImagesPullingTime="7.98057305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:22.412874566 +0000 UTC m=+66.068383377" watchObservedRunningTime="2026-07-23 12:11:22.414481666 +0000 UTC m=+66.069990470"
Jul 23 12:11:22.677621 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.677532    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:11:22.677621 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.677598    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:11:22.680556 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.680512    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 12:11:22.680691 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.680584    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 12:11:22.690918 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.690887    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd0a997-8423-410e-bbda-4cc49e6c94d9-metrics-certs\") pod \"network-metrics-daemon-xb7r6\" (UID: \"ccd0a997-8423-410e-bbda-4cc49e6c94d9\") " pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:11:22.691054 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.690922    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa53ff2f-84ce-4767-8e00-a0230504134b-original-pull-secret\") pod \"global-pull-secret-syncer-f22wm\" (UID: \"aa53ff2f-84ce-4767-8e00-a0230504134b\") " pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:11:22.753477 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.753444    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f22wm"
Jul 23 12:11:22.778396 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.778360    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:11:22.782551 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.782521    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 12:11:22.793672 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.793640    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 12:11:22.802220 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.802164    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7vgf\" (UniqueName: \"kubernetes.io/projected/501c1bcd-51e9-4cc3-9356-ab8ed8de7a87-kube-api-access-t7vgf\") pod \"network-check-target-tkw2s\" (UID: \"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87\") " pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:11:22.920112 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.920075    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f22wm"]
Jul 23 12:11:22.924800 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:11:22.924750    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa53ff2f_84ce_4767_8e00_a0230504134b.slice/crio-0cf77b439b34783a5dcdc68534af59a6f3d9a139e0e5456d55e6be181134951b WatchSource:0}: Error finding container 0cf77b439b34783a5dcdc68534af59a6f3d9a139e0e5456d55e6be181134951b: Status 404 returned error can't find the container with id 0cf77b439b34783a5dcdc68534af59a6f3d9a139e0e5456d55e6be181134951b
Jul 23 12:11:22.958987 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.958957    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pbg55\""
Jul 23 12:11:22.963843 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.963820    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5z96\""
Jul 23 12:11:22.966346 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.966327    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:11:22.971059 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:22.971038    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xb7r6"
Jul 23 12:11:23.120135 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:23.120102    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xb7r6"]
Jul 23 12:11:23.124111 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:11:23.124084    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd0a997_8423_410e_bbda_4cc49e6c94d9.slice/crio-df07a0ae3e0abd5ce9576694599ec78ac15a6568041161cd72c1460426481d29 WatchSource:0}: Error finding container df07a0ae3e0abd5ce9576694599ec78ac15a6568041161cd72c1460426481d29: Status 404 returned error can't find the container with id df07a0ae3e0abd5ce9576694599ec78ac15a6568041161cd72c1460426481d29
Jul 23 12:11:23.138401 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:23.138356    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tkw2s"]
Jul 23 12:11:23.141686 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:11:23.141654    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501c1bcd_51e9_4cc3_9356_ab8ed8de7a87.slice/crio-9c15b1ad6b32a5d0fe4db9144fa85c4bb08c0e38a8a9b2190f707bb249e36167 WatchSource:0}: Error finding container 9c15b1ad6b32a5d0fe4db9144fa85c4bb08c0e38a8a9b2190f707bb249e36167: Status 404 returned error can't find the container with id 9c15b1ad6b32a5d0fe4db9144fa85c4bb08c0e38a8a9b2190f707bb249e36167
Jul 23 12:11:23.397469 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:23.397433    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f22wm" event={"ID":"aa53ff2f-84ce-4767-8e00-a0230504134b","Type":"ContainerStarted","Data":"0cf77b439b34783a5dcdc68534af59a6f3d9a139e0e5456d55e6be181134951b"}
Jul 23 12:11:23.398773 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:23.398724    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xb7r6" event={"ID":"ccd0a997-8423-410e-bbda-4cc49e6c94d9","Type":"ContainerStarted","Data":"df07a0ae3e0abd5ce9576694599ec78ac15a6568041161cd72c1460426481d29"}
Jul 23 12:11:23.399820 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:23.399795    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tkw2s" event={"ID":"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87","Type":"ContainerStarted","Data":"9c15b1ad6b32a5d0fe4db9144fa85c4bb08c0e38a8a9b2190f707bb249e36167"}
Jul 23 12:11:26.415051 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:26.415009    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vsx98" event={"ID":"008cf23c-b927-426d-9a6a-e69d58a5ba3b","Type":"ContainerStarted","Data":"2d7c30eea93052331b58a0b4a19f479400305c8c2e8371604e0ce73559984b1a"}
Jul 23 12:11:26.436553 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:26.436494    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vsx98" podStartSLOduration=17.91703086 podStartE2EDuration="35.436474848s" totalImagesPullingTime="17.519443988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:26.435892474 +0000 UTC m=+70.091401281" watchObservedRunningTime="2026-07-23 12:11:26.436474848 +0000 UTC m=+70.091983650"
Jul 23 12:11:27.421238 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:27.421171    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xb7r6" event={"ID":"ccd0a997-8423-410e-bbda-4cc49e6c94d9","Type":"ContainerStarted","Data":"20801b369257140a9cf2619cdd7c49a1527d48466372eb66898c1db9e4e4956a"}
Jul 23 12:11:27.421713 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:27.421247    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xb7r6" event={"ID":"ccd0a997-8423-410e-bbda-4cc49e6c94d9","Type":"ContainerStarted","Data":"b2829c15662d2dc0f18222a56435e9ed4dffb6fc45528906da4bb71343840505"}
Jul 23 12:11:27.441797 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:27.441722    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xb7r6" podStartSLOduration=67.239706107 podStartE2EDuration="1m10.441700109s" totalImagesPullingTime="3.201994002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:27.441031029 +0000 UTC m=+71.096539835" watchObservedRunningTime="2026-07-23 12:11:27.441700109 +0000 UTC m=+71.097208914"
Jul 23 12:11:29.052240 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.052191    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h6kfl"]
Jul 23 12:11:29.054701 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.054679    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.057797 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.057742    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 12:11:29.059182 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.059140    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sc6d5\""
Jul 23 12:11:29.059296 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.059145    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 12:11:29.059296 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.059207    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 12:11:29.059296 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.059230    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 12:11:29.059296 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.059232    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 12:11:29.059512 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.059148    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 12:11:29.125680 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.125647    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-textfile\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.125884 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.125699    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f918b10a-19fc-4a4e-a932-2631b9cd6c53-root\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.125884 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.125753    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-tls\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.125884 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.125809    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f918b10a-19fc-4a4e-a932-2631b9cd6c53-sys\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.125884 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.125832    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-wtmp\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.126077 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.125908    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.126077 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.125977    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tctp8\" (UniqueName: \"kubernetes.io/projected/f918b10a-19fc-4a4e-a932-2631b9cd6c53-kube-api-access-tctp8\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.126077 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.126006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f918b10a-19fc-4a4e-a932-2631b9cd6c53-metrics-client-ca\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.126077 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.126030    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.226954 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.226917    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-textfile\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227132 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.226978    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f918b10a-19fc-4a4e-a932-2631b9cd6c53-root\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227132 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227011    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-tls\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227132 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227037    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f918b10a-19fc-4a4e-a932-2631b9cd6c53-sys\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227132 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227060    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-wtmp\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227132 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227084    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227132 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f918b10a-19fc-4a4e-a932-2631b9cd6c53-root\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227132 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227128    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tctp8\" (UniqueName: \"kubernetes.io/projected/f918b10a-19fc-4a4e-a932-2631b9cd6c53-kube-api-access-tctp8\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227478 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227154    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f918b10a-19fc-4a4e-a932-2631b9cd6c53-metrics-client-ca\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227478 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227177    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227478 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227198    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f918b10a-19fc-4a4e-a932-2631b9cd6c53-sys\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227478 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227254    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-wtmp\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227478 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227285    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-textfile\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227755 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227691    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f918b10a-19fc-4a4e-a932-2631b9cd6c53-metrics-client-ca\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.227843 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.227821    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.229749 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.229725    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.230018 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.229998    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f918b10a-19fc-4a4e-a932-2631b9cd6c53-node-exporter-tls\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.237630 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.237609    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tctp8\" (UniqueName: \"kubernetes.io/projected/f918b10a-19fc-4a4e-a932-2631b9cd6c53-kube-api-access-tctp8\") pod \"node-exporter-h6kfl\" (UID: \"f918b10a-19fc-4a4e-a932-2631b9cd6c53\") " pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.366698 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:29.366661    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6kfl"
Jul 23 12:11:29.628531 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:11:29.628495    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf918b10a_19fc_4a4e_a932_2631b9cd6c53.slice/crio-9481b9e77ff76ce251c698e9250c3e0acb62ef1a2eb5894e718b2e2867feafe9 WatchSource:0}: Error finding container 9481b9e77ff76ce251c698e9250c3e0acb62ef1a2eb5894e718b2e2867feafe9: Status 404 returned error can't find the container with id 9481b9e77ff76ce251c698e9250c3e0acb62ef1a2eb5894e718b2e2867feafe9
Jul 23 12:11:30.435237 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:30.435202    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tkw2s" event={"ID":"501c1bcd-51e9-4cc3-9356-ab8ed8de7a87","Type":"ContainerStarted","Data":"bf7fb688a081f4e1a1fb4c6aa8c2326e76818b91883bb96dd28a38a22af6412e"}
Jul 23 12:11:30.435649 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:30.435324    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:11:30.436754 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:30.436731    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f22wm" event={"ID":"aa53ff2f-84ce-4767-8e00-a0230504134b","Type":"ContainerStarted","Data":"360d3a10f0dcfb31ad0f2f3e1affb0243fd83cc9f5b6cf817ad1fb8518deab77"}
Jul 23 12:11:30.437861 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:30.437841    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6kfl" event={"ID":"f918b10a-19fc-4a4e-a932-2631b9cd6c53","Type":"ContainerStarted","Data":"9481b9e77ff76ce251c698e9250c3e0acb62ef1a2eb5894e718b2e2867feafe9"}
Jul 23 12:11:30.471401 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:30.470927    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tkw2s" podStartSLOduration=66.980162372 podStartE2EDuration="1m13.470907194s" totalImagesPullingTime="6.490744822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:30.4652985 +0000 UTC m=+74.120807314" watchObservedRunningTime="2026-07-23 12:11:30.470907194 +0000 UTC m=+74.126416001"
Jul 23 12:11:30.517466 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:30.517419    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f22wm" podStartSLOduration=66.801568693 podStartE2EDuration="1m13.517400615s" totalImagesPullingTime="6.715831922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:30.516798429 +0000 UTC m=+74.172307236" watchObservedRunningTime="2026-07-23 12:11:30.517400615 +0000 UTC m=+74.172909419"
Jul 23 12:11:31.443060 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:31.443026    2565 generic.go:356] "Generic (PLEG): container finished" podID="f918b10a-19fc-4a4e-a932-2631b9cd6c53" containerID="713ef37f88153551343c0ca26a0472ca455e201b5d6561949ba5cf6fbcba30aa" exitCode=0
Jul 23 12:11:31.443505 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:31.443126    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6kfl" event={"ID":"f918b10a-19fc-4a4e-a932-2631b9cd6c53","Type":"ContainerDied","Data":"713ef37f88153551343c0ca26a0472ca455e201b5d6561949ba5cf6fbcba30aa"}
Jul 23 12:11:32.449492 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:32.449462    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6kfl" event={"ID":"f918b10a-19fc-4a4e-a932-2631b9cd6c53","Type":"ContainerStarted","Data":"c5b97992d801f16072c0e5f76e21a2d345fbbe01bf4ed1c8c9c2cceeac3fbaa2"}
Jul 23 12:11:32.449898 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:32.449516    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6kfl" event={"ID":"f918b10a-19fc-4a4e-a932-2631b9cd6c53","Type":"ContainerStarted","Data":"a471b6871fd3b498209aa5f9ce99739931fbd64dddc7a985fbc05261b2489e73"}
Jul 23 12:11:32.474403 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:11:32.474355    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h6kfl" podStartSLOduration=2.579143035 podStartE2EDuration="3.474338405s" totalImagesPullingTime="831.979627ms" totalInitContainerRuntime="63.215743ms" isStatefulPod=true podCreationTimestamp="2026-07-23 12:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:32.472665508 +0000 UTC m=+76.128174313" watchObservedRunningTime="2026-07-23 12:11:32.474338405 +0000 UTC m=+76.129847208"
Jul 23 12:12:01.446326 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:12:01.446295    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tkw2s"
Jul 23 12:14:22.837529 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:22.837499    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-stl82"]
Jul 23 12:14:22.839608 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:22.839588    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82"
Jul 23 12:14:22.842495 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:22.842476    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9x8dq\""
Jul 23 12:14:22.843712 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:22.843694    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 12:14:22.843832 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:22.843719    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 12:14:22.851709 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:22.851687    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-stl82"]
Jul 23 12:14:22.912816 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:22.912744    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8chm5\" (UniqueName: \"kubernetes.io/projected/48dbfd96-d675-4f19-b400-3e8e57e423ac-kube-api-access-8chm5\") pod \"cert-manager-cainjector-559cdd8c87-stl82\" (UID: \"48dbfd96-d675-4f19-b400-3e8e57e423ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82"
Jul 23 12:14:22.913001 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:22.912869    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48dbfd96-d675-4f19-b400-3e8e57e423ac-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-stl82\" (UID: \"48dbfd96-d675-4f19-b400-3e8e57e423ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82"
Jul 23 12:14:23.014085 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:23.014048    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8chm5\" (UniqueName: \"kubernetes.io/projected/48dbfd96-d675-4f19-b400-3e8e57e423ac-kube-api-access-8chm5\") pod \"cert-manager-cainjector-559cdd8c87-stl82\" (UID: \"48dbfd96-d675-4f19-b400-3e8e57e423ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82"
Jul 23 12:14:23.014284 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:23.014112    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48dbfd96-d675-4f19-b400-3e8e57e423ac-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-stl82\" (UID: \"48dbfd96-d675-4f19-b400-3e8e57e423ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82"
Jul 23 12:14:23.024265 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:23.024237    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48dbfd96-d675-4f19-b400-3e8e57e423ac-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-stl82\" (UID: \"48dbfd96-d675-4f19-b400-3e8e57e423ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82"
Jul 23 12:14:23.024400 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:23.024383    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8chm5\" (UniqueName: \"kubernetes.io/projected/48dbfd96-d675-4f19-b400-3e8e57e423ac-kube-api-access-8chm5\") pod \"cert-manager-cainjector-559cdd8c87-stl82\" (UID: \"48dbfd96-d675-4f19-b400-3e8e57e423ac\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82"
Jul 23 12:14:23.148136 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:23.148047    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82"
Jul 23 12:14:23.274375 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:23.274336    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-stl82"]
Jul 23 12:14:23.277667 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:14:23.277639    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48dbfd96_d675_4f19_b400_3e8e57e423ac.slice/crio-dcd930e7f015fab7135bf838bdf8bbfdf67ee7ad216ceb03faf0782df1abdfd0 WatchSource:0}: Error finding container dcd930e7f015fab7135bf838bdf8bbfdf67ee7ad216ceb03faf0782df1abdfd0: Status 404 returned error can't find the container with id dcd930e7f015fab7135bf838bdf8bbfdf67ee7ad216ceb03faf0782df1abdfd0
Jul 23 12:14:23.997710 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:23.997666    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82" event={"ID":"48dbfd96-d675-4f19-b400-3e8e57e423ac","Type":"ContainerStarted","Data":"dcd930e7f015fab7135bf838bdf8bbfdf67ee7ad216ceb03faf0782df1abdfd0"}
Jul 23 12:14:27.008986 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:27.008952    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82" event={"ID":"48dbfd96-d675-4f19-b400-3e8e57e423ac","Type":"ContainerStarted","Data":"57f95d0eb310d37fd4b1c0b3ab7119b52e226f9aa67004d74356b79254e07c93"}
Jul 23 12:14:27.046780 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:27.046712    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-stl82" podStartSLOduration=2.162794905 podStartE2EDuration="5.046694545s" totalImagesPullingTime="2.88389964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:14:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:14:27.046214238 +0000 UTC m=+250.701723036" watchObservedRunningTime="2026-07-23 12:14:27.046694545 +0000 UTC m=+250.702203350"
Jul 23 12:14:56.391988 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.391954    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"]
Jul 23 12:14:56.396434 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.396415    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.400337 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.400296    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 12:14:56.400337 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.400317    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 12:14:56.400337 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.400331    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 12:14:56.400598 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.400360    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 12:14:56.400598 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.400334    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4zrkc\""
Jul 23 12:14:56.415276 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.415252    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"]
Jul 23 12:14:56.559582 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.559551    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/52116d9e-e158-442e-a64c-a6e7b02d0973-manifests\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.559805 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.559592    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/52116d9e-e158-442e-a64c-a6e7b02d0973-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.559805 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.559611    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52116d9e-e158-442e-a64c-a6e7b02d0973-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.559805 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.559630    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52116d9e-e158-442e-a64c-a6e7b02d0973-tmp\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.559805 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.559669    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vclk\" (UniqueName: \"kubernetes.io/projected/52116d9e-e158-442e-a64c-a6e7b02d0973-kube-api-access-8vclk\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.661032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.660949    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/52116d9e-e158-442e-a64c-a6e7b02d0973-manifests\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.661032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.661005    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/52116d9e-e158-442e-a64c-a6e7b02d0973-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.661032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.661022    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52116d9e-e158-442e-a64c-a6e7b02d0973-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.661273 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.661042    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52116d9e-e158-442e-a64c-a6e7b02d0973-tmp\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.661273 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.661090    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vclk\" (UniqueName: \"kubernetes.io/projected/52116d9e-e158-442e-a64c-a6e7b02d0973-kube-api-access-8vclk\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.661521 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.661474    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/52116d9e-e158-442e-a64c-a6e7b02d0973-manifests\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.663375 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.663351    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52116d9e-e158-442e-a64c-a6e7b02d0973-tmp\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.663472 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.663421    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52116d9e-e158-442e-a64c-a6e7b02d0973-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.663653 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.663634    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/52116d9e-e158-442e-a64c-a6e7b02d0973-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.671299 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.671270    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vclk\" (UniqueName: \"kubernetes.io/projected/52116d9e-e158-442e-a64c-a6e7b02d0973-kube-api-access-8vclk\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b\" (UID: \"52116d9e-e158-442e-a64c-a6e7b02d0973\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.706166 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.706129    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:14:56.847743 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:56.847719    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"]
Jul 23 12:14:56.850115 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:14:56.850094    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52116d9e_e158_442e_a64c_a6e7b02d0973.slice/crio-60892f2bfa8a18eca10f59fc770f2944d4c6be1ea38d79b8782b0652eb6762ea WatchSource:0}: Error finding container 60892f2bfa8a18eca10f59fc770f2944d4c6be1ea38d79b8782b0652eb6762ea: Status 404 returned error can't find the container with id 60892f2bfa8a18eca10f59fc770f2944d4c6be1ea38d79b8782b0652eb6762ea
Jul 23 12:14:57.104340 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:14:57.104301    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b" event={"ID":"52116d9e-e158-442e-a64c-a6e7b02d0973","Type":"ContainerStarted","Data":"60892f2bfa8a18eca10f59fc770f2944d4c6be1ea38d79b8782b0652eb6762ea"}
Jul 23 12:15:01.119996 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:01.119958    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b" event={"ID":"52116d9e-e158-442e-a64c-a6e7b02d0973","Type":"ContainerStarted","Data":"61c48445291c3b2d3e8cea687981356b03b638b89d3cba124f723dad63deb9ff"}
Jul 23 12:15:02.124328 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:02.124299    2565 generic.go:356] "Generic (PLEG): container finished" podID="52116d9e-e158-442e-a64c-a6e7b02d0973" containerID="61c48445291c3b2d3e8cea687981356b03b638b89d3cba124f723dad63deb9ff" exitCode=0
Jul 23 12:15:02.124707 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:02.124357    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b" event={"ID":"52116d9e-e158-442e-a64c-a6e7b02d0973","Type":"ContainerDied","Data":"61c48445291c3b2d3e8cea687981356b03b638b89d3cba124f723dad63deb9ff"}
Jul 23 12:15:03.129433 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:03.129399    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b" event={"ID":"52116d9e-e158-442e-a64c-a6e7b02d0973","Type":"ContainerStarted","Data":"68013ca8958bcf4edf22e2b854e9d423a2a3149fa47dc30a022b0178d236816b"}
Jul 23 12:15:03.129843 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:03.129507    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:15:03.156830 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:03.156782    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b" podStartSLOduration=2.437731266 podStartE2EDuration="7.156746532s" totalImagesPullingTime="3.716963793s" totalInitContainerRuntime="1.002051473s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:14:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:15:03.1553441 +0000 UTC m=+286.810852903" watchObservedRunningTime="2026-07-23 12:15:03.156746532 +0000 UTC m=+286.812255336"
Jul 23 12:15:14.137920 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:14.137887    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b"
Jul 23 12:15:16.871752 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:16.871725    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 12:15:17.864446 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.864406    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"]
Jul 23 12:15:17.866388 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.866372    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:17.870316 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.870295    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 12:15:17.870432 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.870332    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 12:15:17.870522 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.870508    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 12:15:17.873505 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.873482    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 12:15:17.893609 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.893581    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"]
Jul 23 12:15:17.908518 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.908497    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hv88\" (UniqueName: \"kubernetes.io/projected/fbc1ee4a-41f8-454d-9058-8035ca02e135-kube-api-access-6hv88\") pod \"kube-auth-proxy-5f7c78d7c4-j7pjt\" (UID: \"fbc1ee4a-41f8-454d-9058-8035ca02e135\") " pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:17.908640 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.908540    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc1ee4a-41f8-454d-9058-8035ca02e135-tls-certs\") pod \"kube-auth-proxy-5f7c78d7c4-j7pjt\" (UID: \"fbc1ee4a-41f8-454d-9058-8035ca02e135\") " pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:17.908640 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:17.908565    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbc1ee4a-41f8-454d-9058-8035ca02e135-tmp\") pod \"kube-auth-proxy-5f7c78d7c4-j7pjt\" (UID: \"fbc1ee4a-41f8-454d-9058-8035ca02e135\") " pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:18.009212 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.009180    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hv88\" (UniqueName: \"kubernetes.io/projected/fbc1ee4a-41f8-454d-9058-8035ca02e135-kube-api-access-6hv88\") pod \"kube-auth-proxy-5f7c78d7c4-j7pjt\" (UID: \"fbc1ee4a-41f8-454d-9058-8035ca02e135\") " pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:18.009405 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.009259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc1ee4a-41f8-454d-9058-8035ca02e135-tls-certs\") pod \"kube-auth-proxy-5f7c78d7c4-j7pjt\" (UID: \"fbc1ee4a-41f8-454d-9058-8035ca02e135\") " pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:18.009405 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.009287    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbc1ee4a-41f8-454d-9058-8035ca02e135-tmp\") pod \"kube-auth-proxy-5f7c78d7c4-j7pjt\" (UID: \"fbc1ee4a-41f8-454d-9058-8035ca02e135\") " pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:18.011483 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.011460    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbc1ee4a-41f8-454d-9058-8035ca02e135-tmp\") pod \"kube-auth-proxy-5f7c78d7c4-j7pjt\" (UID: \"fbc1ee4a-41f8-454d-9058-8035ca02e135\") " pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:18.011682 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.011662    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbc1ee4a-41f8-454d-9058-8035ca02e135-tls-certs\") pod \"kube-auth-proxy-5f7c78d7c4-j7pjt\" (UID: \"fbc1ee4a-41f8-454d-9058-8035ca02e135\") " pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:18.032586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.032552    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hv88\" (UniqueName: \"kubernetes.io/projected/fbc1ee4a-41f8-454d-9058-8035ca02e135-kube-api-access-6hv88\") pod \"kube-auth-proxy-5f7c78d7c4-j7pjt\" (UID: \"fbc1ee4a-41f8-454d-9058-8035ca02e135\") " pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:18.175555 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.175477    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"
Jul 23 12:15:18.306947 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.306916    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt"]
Jul 23 12:15:18.309703 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:15:18.309678    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc1ee4a_41f8_454d_9058_8035ca02e135.slice/crio-8a2a34d2007bfc74449be43ec3fe68345234d3c36259cf64a4f0b7e60fdc3d2a WatchSource:0}: Error finding container 8a2a34d2007bfc74449be43ec3fe68345234d3c36259cf64a4f0b7e60fdc3d2a: Status 404 returned error can't find the container with id 8a2a34d2007bfc74449be43ec3fe68345234d3c36259cf64a4f0b7e60fdc3d2a
Jul 23 12:15:18.880505 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.880473    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4pptf"]
Jul 23 12:15:18.883064 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.883048    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:18.887817 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.887799    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-dfzlp\""
Jul 23 12:15:18.888881 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.888856    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 23 12:15:18.903037 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.903006    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4pptf"]
Jul 23 12:15:18.915777 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.915738    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg27j\" (UniqueName: \"kubernetes.io/projected/5e4c8a7e-873f-4d58-aa64-178cae6dd57d-kube-api-access-mg27j\") pod \"model-serving-api-54d89bd5fc-4pptf\" (UID: \"5e4c8a7e-873f-4d58-aa64-178cae6dd57d\") " pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:18.915921 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:18.915800    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e4c8a7e-873f-4d58-aa64-178cae6dd57d-tls-certs\") pod \"model-serving-api-54d89bd5fc-4pptf\" (UID: \"5e4c8a7e-873f-4d58-aa64-178cae6dd57d\") " pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:19.016513 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.016476    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg27j\" (UniqueName: \"kubernetes.io/projected/5e4c8a7e-873f-4d58-aa64-178cae6dd57d-kube-api-access-mg27j\") pod \"model-serving-api-54d89bd5fc-4pptf\" (UID: \"5e4c8a7e-873f-4d58-aa64-178cae6dd57d\") " pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:19.016693 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.016530    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e4c8a7e-873f-4d58-aa64-178cae6dd57d-tls-certs\") pod \"model-serving-api-54d89bd5fc-4pptf\" (UID: \"5e4c8a7e-873f-4d58-aa64-178cae6dd57d\") " pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:19.018898 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.018878    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e4c8a7e-873f-4d58-aa64-178cae6dd57d-tls-certs\") pod \"model-serving-api-54d89bd5fc-4pptf\" (UID: \"5e4c8a7e-873f-4d58-aa64-178cae6dd57d\") " pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:19.026857 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.026837    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg27j\" (UniqueName: \"kubernetes.io/projected/5e4c8a7e-873f-4d58-aa64-178cae6dd57d-kube-api-access-mg27j\") pod \"model-serving-api-54d89bd5fc-4pptf\" (UID: \"5e4c8a7e-873f-4d58-aa64-178cae6dd57d\") " pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:19.187644 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.187553    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt" event={"ID":"fbc1ee4a-41f8-454d-9058-8035ca02e135","Type":"ContainerStarted","Data":"8a2a34d2007bfc74449be43ec3fe68345234d3c36259cf64a4f0b7e60fdc3d2a"}
Jul 23 12:15:19.192900 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.192871    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:19.257636 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.257608    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"]
Jul 23 12:15:19.260195 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.260173    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.262980 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.262957    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 12:15:19.263233 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.263189    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 23 12:15:19.263233 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.263206    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 23 12:15:19.263233 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.263209    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-z5gd8\""
Jul 23 12:15:19.263481 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.263450    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 23 12:15:19.278528 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.278508    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"]
Jul 23 12:15:19.319395 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.319360    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-local-certs\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.319566 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.319402    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.319566 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.319446    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.319566 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.319480    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.319566 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.319511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-token\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.319566 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.319534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-cacerts\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.319753 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.319646    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw5s7\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-kube-api-access-tw5s7\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.334285 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.334258    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4pptf"]
Jul 23 12:15:19.338587 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:15:19.338561    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4c8a7e_873f_4d58_aa64_178cae6dd57d.slice/crio-500ff4f65be025bc132d7f00c36788c1b3330507e14c961893f4991ff0fec951 WatchSource:0}: Error finding container 500ff4f65be025bc132d7f00c36788c1b3330507e14c961893f4991ff0fec951: Status 404 returned error can't find the container with id 500ff4f65be025bc132d7f00c36788c1b3330507e14c961893f4991ff0fec951
Jul 23 12:15:19.340988 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.340966    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 12:15:19.420098 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.420068    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-token\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.420291 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.420105    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-cacerts\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.420291 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.420237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5s7\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-kube-api-access-tw5s7\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.420291 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.420284    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-local-certs\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.420457 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.420308    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.420457 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.420363    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.420457 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.420412    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.421183 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.421154    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.423174 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.423147    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.423315 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.423189    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.423315 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.423212    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-local-certs\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.423315 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.423257    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-cacerts\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.435223 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.435189    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-token\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.436634 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.436615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw5s7\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-kube-api-access-tw5s7\") pod \"istiod-openshift-gateway-7574459656-pmp68\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.569804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.569775    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:19.710259 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:19.710221    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"]
Jul 23 12:15:19.714213 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:15:19.714176    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5309bfba_fcae_4ab2_bcf5_33bb3afd0fc7.slice/crio-7db5a5cb4963cbabca4a2fc8ef072b1167b9a6cb1162dce00c59ae3fd6ef4d80 WatchSource:0}: Error finding container 7db5a5cb4963cbabca4a2fc8ef072b1167b9a6cb1162dce00c59ae3fd6ef4d80: Status 404 returned error can't find the container with id 7db5a5cb4963cbabca4a2fc8ef072b1167b9a6cb1162dce00c59ae3fd6ef4d80
Jul 23 12:15:20.192777 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:20.192732    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4pptf" event={"ID":"5e4c8a7e-873f-4d58-aa64-178cae6dd57d","Type":"ContainerStarted","Data":"500ff4f65be025bc132d7f00c36788c1b3330507e14c961893f4991ff0fec951"}
Jul 23 12:15:20.194176 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:20.194141    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68" event={"ID":"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7","Type":"ContainerStarted","Data":"7db5a5cb4963cbabca4a2fc8ef072b1167b9a6cb1162dce00c59ae3fd6ef4d80"}
Jul 23 12:15:22.799409 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:22.799328    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 12:15:22.799792 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:22.799419    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 12:15:23.208513 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.208477    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4pptf" event={"ID":"5e4c8a7e-873f-4d58-aa64-178cae6dd57d","Type":"ContainerStarted","Data":"9ac5d00cfc8401feff9acb575a7fba2e73395504b383a57d2f11ed6b3e317a57"}
Jul 23 12:15:23.208717 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.208563    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:23.210128 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.210104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt" event={"ID":"fbc1ee4a-41f8-454d-9058-8035ca02e135","Type":"ContainerStarted","Data":"a18623f4ce84e74304dcbaa8b9bb66f5e530b8e0350d4b6a85ff850b6990fc18"}
Jul 23 12:15:23.211590 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.211571    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68" event={"ID":"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7","Type":"ContainerStarted","Data":"58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d"}
Jul 23 12:15:23.211778 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.211749    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:23.213324 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.213300    2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-pmp68 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 12:15:23.213414 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.213351    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68" podUID="5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 12:15:23.232445 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.232398    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4pptf" podStartSLOduration=1.779818787 podStartE2EDuration="5.232384938s" totalImagesPullingTime="3.452566151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:15:23.231086626 +0000 UTC m=+306.886595431" watchObservedRunningTime="2026-07-23 12:15:23.232384938 +0000 UTC m=+306.887893741"
Jul 23 12:15:23.251189 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.251149    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f7c78d7c4-j7pjt" podStartSLOduration=1.769768527 podStartE2EDuration="6.251138366s" totalImagesPullingTime="4.481369839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:15:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:15:23.250779865 +0000 UTC m=+306.906288663" watchObservedRunningTime="2026-07-23 12:15:23.251138366 +0000 UTC m=+306.906647225"
Jul 23 12:15:23.275491 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:23.275440    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68" podStartSLOduration=1.19274009 podStartE2EDuration="4.275425187s" totalImagesPullingTime="3.082685097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:15:23.273281594 +0000 UTC m=+306.928790399" watchObservedRunningTime="2026-07-23 12:15:23.275425187 +0000 UTC m=+306.930933992"
Jul 23 12:15:24.215251 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:24.215222    2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-pmp68 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 12:15:24.215605 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:24.215279    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68" podUID="5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 12:15:27.215550 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:27.215513    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:29.015230 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.015197    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"]
Jul 23 12:15:29.018663 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.018643    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:29.022048 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.022029    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 23 12:15:29.022137 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.022093    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sjhw7\""
Jul 23 12:15:29.032188 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.032159    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"]
Jul 23 12:15:29.033882 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.033863    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"]
Jul 23 12:15:29.036987 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.036970    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:29.042239 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.042216    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-b7xfs\""
Jul 23 12:15:29.043319 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.043301    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 23 12:15:29.048685 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.048662    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"]
Jul 23 12:15:29.091041 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.091005    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gg2d\" (UniqueName: \"kubernetes.io/projected/1aa67db6-d44a-4399-ba23-ce61cffb25ac-kube-api-access-2gg2d\") pod \"llmisvc-controller-manager-777c6bd7d5-m7cmn\" (UID: \"1aa67db6-d44a-4399-ba23-ce61cffb25ac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:29.091229 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.091049    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1aa67db6-d44a-4399-ba23-ce61cffb25ac-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m7cmn\" (UID: \"1aa67db6-d44a-4399-ba23-ce61cffb25ac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:29.091229 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.091068    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqs7c\" (UniqueName: \"kubernetes.io/projected/c3544862-f020-4982-abbc-32788f9c8649-kube-api-access-lqs7c\") pod \"kserve-controller-manager-d6f4b65b8-xhlcv\" (UID: \"c3544862-f020-4982-abbc-32788f9c8649\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:29.091229 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.091156    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3544862-f020-4982-abbc-32788f9c8649-cert\") pod \"kserve-controller-manager-d6f4b65b8-xhlcv\" (UID: \"c3544862-f020-4982-abbc-32788f9c8649\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:29.191635 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.191604    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3544862-f020-4982-abbc-32788f9c8649-cert\") pod \"kserve-controller-manager-d6f4b65b8-xhlcv\" (UID: \"c3544862-f020-4982-abbc-32788f9c8649\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:29.191844 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.191650    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gg2d\" (UniqueName: \"kubernetes.io/projected/1aa67db6-d44a-4399-ba23-ce61cffb25ac-kube-api-access-2gg2d\") pod \"llmisvc-controller-manager-777c6bd7d5-m7cmn\" (UID: \"1aa67db6-d44a-4399-ba23-ce61cffb25ac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:29.191844 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.191682    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1aa67db6-d44a-4399-ba23-ce61cffb25ac-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m7cmn\" (UID: \"1aa67db6-d44a-4399-ba23-ce61cffb25ac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:29.191844 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.191700    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs7c\" (UniqueName: \"kubernetes.io/projected/c3544862-f020-4982-abbc-32788f9c8649-kube-api-access-lqs7c\") pod \"kserve-controller-manager-d6f4b65b8-xhlcv\" (UID: \"c3544862-f020-4982-abbc-32788f9c8649\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:29.191844 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:15:29.191807    2565 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 23 12:15:29.191988 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:15:29.191889    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aa67db6-d44a-4399-ba23-ce61cffb25ac-cert podName:1aa67db6-d44a-4399-ba23-ce61cffb25ac nodeName:}" failed. No retries permitted until 2026-07-23 12:15:29.691867066 +0000 UTC m=+313.347375854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1aa67db6-d44a-4399-ba23-ce61cffb25ac-cert") pod "llmisvc-controller-manager-777c6bd7d5-m7cmn" (UID: "1aa67db6-d44a-4399-ba23-ce61cffb25ac") : secret "llmisvc-webhook-server-cert" not found
Jul 23 12:15:29.191988 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:15:29.191807    2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 23 12:15:29.191988 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:15:29.191951    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3544862-f020-4982-abbc-32788f9c8649-cert podName:c3544862-f020-4982-abbc-32788f9c8649 nodeName:}" failed. No retries permitted until 2026-07-23 12:15:29.691938209 +0000 UTC m=+313.347446992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c3544862-f020-4982-abbc-32788f9c8649-cert") pod "kserve-controller-manager-d6f4b65b8-xhlcv" (UID: "c3544862-f020-4982-abbc-32788f9c8649") : secret "kserve-webhook-server-cert" not found
Jul 23 12:15:29.203657 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.203625    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gg2d\" (UniqueName: \"kubernetes.io/projected/1aa67db6-d44a-4399-ba23-ce61cffb25ac-kube-api-access-2gg2d\") pod \"llmisvc-controller-manager-777c6bd7d5-m7cmn\" (UID: \"1aa67db6-d44a-4399-ba23-ce61cffb25ac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:29.211047 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.211021    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqs7c\" (UniqueName: \"kubernetes.io/projected/c3544862-f020-4982-abbc-32788f9c8649-kube-api-access-lqs7c\") pod \"kserve-controller-manager-d6f4b65b8-xhlcv\" (UID: \"c3544862-f020-4982-abbc-32788f9c8649\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:29.696241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.696210    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3544862-f020-4982-abbc-32788f9c8649-cert\") pod \"kserve-controller-manager-d6f4b65b8-xhlcv\" (UID: \"c3544862-f020-4982-abbc-32788f9c8649\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:29.696433 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.696263    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1aa67db6-d44a-4399-ba23-ce61cffb25ac-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m7cmn\" (UID: \"1aa67db6-d44a-4399-ba23-ce61cffb25ac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:29.698800 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.698753    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1aa67db6-d44a-4399-ba23-ce61cffb25ac-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-m7cmn\" (UID: \"1aa67db6-d44a-4399-ba23-ce61cffb25ac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:29.698915 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.698892    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3544862-f020-4982-abbc-32788f9c8649-cert\") pod \"kserve-controller-manager-d6f4b65b8-xhlcv\" (UID: \"c3544862-f020-4982-abbc-32788f9c8649\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:29.928934 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.928900    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:29.946093 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:29.946068    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:30.071678 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:30.071635    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"]
Jul 23 12:15:30.075684 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:15:30.075658    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3544862_f020_4982_abbc_32788f9c8649.slice/crio-d1ce3f9bd9c87946c5cc690d199a3cecc664dda52ac6b9cf78fb6b9aefe583f5 WatchSource:0}: Error finding container d1ce3f9bd9c87946c5cc690d199a3cecc664dda52ac6b9cf78fb6b9aefe583f5: Status 404 returned error can't find the container with id d1ce3f9bd9c87946c5cc690d199a3cecc664dda52ac6b9cf78fb6b9aefe583f5
Jul 23 12:15:30.094277 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:30.094249    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"]
Jul 23 12:15:30.096127 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:15:30.096099    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1aa67db6_d44a_4399_ba23_ce61cffb25ac.slice/crio-b2ef71f83335f28a91b5ab08bf0e7f504999f1e35569007777496efe73f00e68 WatchSource:0}: Error finding container b2ef71f83335f28a91b5ab08bf0e7f504999f1e35569007777496efe73f00e68: Status 404 returned error can't find the container with id b2ef71f83335f28a91b5ab08bf0e7f504999f1e35569007777496efe73f00e68
Jul 23 12:15:30.236363 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:30.236269    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn" event={"ID":"1aa67db6-d44a-4399-ba23-ce61cffb25ac","Type":"ContainerStarted","Data":"b2ef71f83335f28a91b5ab08bf0e7f504999f1e35569007777496efe73f00e68"}
Jul 23 12:15:30.237342 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:30.237319    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv" event={"ID":"c3544862-f020-4982-abbc-32788f9c8649","Type":"ContainerStarted","Data":"d1ce3f9bd9c87946c5cc690d199a3cecc664dda52ac6b9cf78fb6b9aefe583f5"}
Jul 23 12:15:31.882372 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:31.882332    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"]
Jul 23 12:15:31.882955 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:31.882557    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68" podUID="5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" containerName="discovery" containerID="cri-o://58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d" gracePeriod=30
Jul 23 12:15:32.145833 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.145813    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:32.221048 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.221017    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-dns-cert\") pod \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") "
Jul 23 12:15:32.221048 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.221051    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-ca-configmap\") pod \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") "
Jul 23 12:15:32.221299 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.221078    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-local-certs\") pod \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") "
Jul 23 12:15:32.221299 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.221115    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-token\") pod \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") "
Jul 23 12:15:32.221299 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.221142    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-cacerts\") pod \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") "
Jul 23 12:15:32.221299 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.221171    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw5s7\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-kube-api-access-tw5s7\") pod \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") "
Jul 23 12:15:32.221299 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.221213    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-kubeconfig\") pod \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\" (UID: \"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7\") "
Jul 23 12:15:32.221545 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.221508    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" (UID: "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:15:32.223409 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.223336    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-local-certs" (OuterVolumeSpecName: "local-certs") pod "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" (UID: "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:15:32.223554 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.223448    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-token" (OuterVolumeSpecName: "istio-token") pod "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" (UID: "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:15:32.223643 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.223621    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-cacerts" (OuterVolumeSpecName: "cacerts") pod "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" (UID: "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:15:32.223711 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.223684    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" (UID: "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:15:32.223826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.223735    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" (UID: "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:15:32.223826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.223747    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-kube-api-access-tw5s7" (OuterVolumeSpecName: "kube-api-access-tw5s7") pod "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" (UID: "5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7"). InnerVolumeSpecName "kube-api-access-tw5s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:15:32.247981 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.247943    2565 generic.go:356] "Generic (PLEG): container finished" podID="5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" containerID="58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d" exitCode=0
Jul 23 12:15:32.248168 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.247999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68" event={"ID":"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7","Type":"ContainerDied","Data":"58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d"}
Jul 23 12:15:32.248168 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.248060    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68" event={"ID":"5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7","Type":"ContainerDied","Data":"7db5a5cb4963cbabca4a2fc8ef072b1167b9a6cb1162dce00c59ae3fd6ef4d80"}
Jul 23 12:15:32.248168 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.248088    2565 scope.go:117] "RemoveContainer" containerID="58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d"
Jul 23 12:15:32.248168 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.248015    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"
Jul 23 12:15:32.258929 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.258857    2565 scope.go:117] "RemoveContainer" containerID="58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d"
Jul 23 12:15:32.259398 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:15:32.259320    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d\": container with ID starting with 58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d not found: ID does not exist" containerID="58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d"
Jul 23 12:15:32.259398 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.259363    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d"} err="failed to get container status \"58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d\": rpc error: code = NotFound desc = could not find container \"58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d\": container with ID starting with 58df12b3b76514710d745cf8aa91a7085ee39870155662c0de3662ed40a9227d not found: ID does not exist"
Jul 23 12:15:32.285047 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.285003    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"]
Jul 23 12:15:32.292777 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.292732    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pmp68"]
Jul 23 12:15:32.322620 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.322589    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-token\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:15:32.322620 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.322618    2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-cacerts\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:15:32.322823 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.322633    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw5s7\" (UniqueName: \"kubernetes.io/projected/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-kube-api-access-tw5s7\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:15:32.322823 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.322646    2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-kubeconfig\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:15:32.322823 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.322657    2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-dns-cert\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:15:32.322823 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.322664    2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-istio-csr-ca-configmap\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:15:32.322823 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:32.322672    2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7-local-certs\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:15:33.049602 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:33.049561    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" path="/var/lib/kubelet/pods/5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7/volumes"
Jul 23 12:15:34.218663 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:34.218633    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4pptf"
Jul 23 12:15:34.258587 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:34.258542    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv" event={"ID":"c3544862-f020-4982-abbc-32788f9c8649","Type":"ContainerStarted","Data":"fe9389510c4501b4bc0b6b951056cb5ad19716fbf9f06c600e98f290207431e3"}
Jul 23 12:15:34.258778 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:34.258687    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:15:34.260041 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:34.260019    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn" event={"ID":"1aa67db6-d44a-4399-ba23-ce61cffb25ac","Type":"ContainerStarted","Data":"23dc93fd4806d7409e8c901362db7d10714f6fb006cf03180a21b1238672b5f3"}
Jul 23 12:15:34.260156 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:34.260142    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:15:34.295440 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:15:34.295391    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv" podStartSLOduration=2.832900931 podStartE2EDuration="6.29537318s" totalImagesPullingTime="3.462472249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:15:34.290351698 +0000 UTC m=+317.945860501" watchObservedRunningTime="2026-07-23 12:15:34.29537318 +0000 UTC m=+317.950881984"
Jul 23 12:16:05.266138 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:05.266106    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn"
Jul 23 12:16:05.269109 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:05.269085    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhlcv"
Jul 23 12:16:05.295857 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:05.295786    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-m7cmn" podStartSLOduration=32.849942147 podStartE2EDuration="36.295746103s" totalImagesPullingTime="3.445803956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:15:34.333108345 +0000 UTC m=+317.988617148" watchObservedRunningTime="2026-07-23 12:16:05.295746103 +0000 UTC m=+348.951254907"
Jul 23 12:16:19.012065 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.012030    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hw49s"]
Jul 23 12:16:19.012461 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.012284    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" containerName="discovery"
Jul 23 12:16:19.012461 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.012294    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" containerName="discovery"
Jul 23 12:16:19.012461 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.012358    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5309bfba-fcae-4ab2-bcf5-33bb3afd0fc7" containerName="discovery"
Jul 23 12:16:19.014928 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.014912    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hw49s"
Jul 23 12:16:19.017597 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.017573    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-b5bg5\""
Jul 23 12:16:19.020178 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.020152    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 12:16:19.020178 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.020173    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 12:16:19.048954 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.048925    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hw49s"]
Jul 23 12:16:19.178723 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.178686    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh9cz\" (UniqueName: \"kubernetes.io/projected/c4953609-37ed-4297-bb47-ddfc55ba2fb3-kube-api-access-qh9cz\") pod \"kuadrant-operator-catalog-hw49s\" (UID: \"c4953609-37ed-4297-bb47-ddfc55ba2fb3\") " pod="kuadrant-system/kuadrant-operator-catalog-hw49s"
Jul 23 12:16:19.279991 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.279907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh9cz\" (UniqueName: \"kubernetes.io/projected/c4953609-37ed-4297-bb47-ddfc55ba2fb3-kube-api-access-qh9cz\") pod \"kuadrant-operator-catalog-hw49s\" (UID: \"c4953609-37ed-4297-bb47-ddfc55ba2fb3\") " pod="kuadrant-system/kuadrant-operator-catalog-hw49s"
Jul 23 12:16:19.292807 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.292780    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh9cz\" (UniqueName: \"kubernetes.io/projected/c4953609-37ed-4297-bb47-ddfc55ba2fb3-kube-api-access-qh9cz\") pod \"kuadrant-operator-catalog-hw49s\" (UID: \"c4953609-37ed-4297-bb47-ddfc55ba2fb3\") " pod="kuadrant-system/kuadrant-operator-catalog-hw49s"
Jul 23 12:16:19.324862 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.324832    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hw49s"
Jul 23 12:16:19.330659 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.330635    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hw49s"]
Jul 23 12:16:19.477644 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.477615    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hw49s"]
Jul 23 12:16:19.479462 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:16:19.479436    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4953609_37ed_4297_bb47_ddfc55ba2fb3.slice/crio-dd94f591a99319653cf0a6a46273423f6c4ab58dae0a58cdd86f802ccc3a8fd4 WatchSource:0}: Error finding container dd94f591a99319653cf0a6a46273423f6c4ab58dae0a58cdd86f802ccc3a8fd4: Status 404 returned error can't find the container with id dd94f591a99319653cf0a6a46273423f6c4ab58dae0a58cdd86f802ccc3a8fd4
Jul 23 12:16:19.573809 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.573752    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mkjq8"]
Jul 23 12:16:19.576369 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.576351    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mkjq8"
Jul 23 12:16:19.611744 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.611713    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mkjq8"]
Jul 23 12:16:19.682756 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.682715    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfcrp\" (UniqueName: \"kubernetes.io/projected/a0a054f4-3278-425f-ada5-65f0d5b116f2-kube-api-access-lfcrp\") pod \"kuadrant-operator-catalog-mkjq8\" (UID: \"a0a054f4-3278-425f-ada5-65f0d5b116f2\") " pod="kuadrant-system/kuadrant-operator-catalog-mkjq8"
Jul 23 12:16:19.783588 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.783559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfcrp\" (UniqueName: \"kubernetes.io/projected/a0a054f4-3278-425f-ada5-65f0d5b116f2-kube-api-access-lfcrp\") pod \"kuadrant-operator-catalog-mkjq8\" (UID: \"a0a054f4-3278-425f-ada5-65f0d5b116f2\") " pod="kuadrant-system/kuadrant-operator-catalog-mkjq8"
Jul 23 12:16:19.795645 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.795625    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfcrp\" (UniqueName: \"kubernetes.io/projected/a0a054f4-3278-425f-ada5-65f0d5b116f2-kube-api-access-lfcrp\") pod \"kuadrant-operator-catalog-mkjq8\" (UID: \"a0a054f4-3278-425f-ada5-65f0d5b116f2\") " pod="kuadrant-system/kuadrant-operator-catalog-mkjq8"
Jul 23 12:16:19.885738 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:19.885652    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mkjq8"
Jul 23 12:16:20.016987 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:20.016942    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mkjq8"]
Jul 23 12:16:20.056287 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:16:20.056259    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a054f4_3278_425f_ada5_65f0d5b116f2.slice/crio-27df9e686324240278898d9e0ae31a727436cd34b1ad4e3b33b6ff14e65be401 WatchSource:0}: Error finding container 27df9e686324240278898d9e0ae31a727436cd34b1ad4e3b33b6ff14e65be401: Status 404 returned error can't find the container with id 27df9e686324240278898d9e0ae31a727436cd34b1ad4e3b33b6ff14e65be401
Jul 23 12:16:20.428904 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:20.428869    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mkjq8" event={"ID":"a0a054f4-3278-425f-ada5-65f0d5b116f2","Type":"ContainerStarted","Data":"27df9e686324240278898d9e0ae31a727436cd34b1ad4e3b33b6ff14e65be401"}
Jul 23 12:16:20.430069 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:20.430042    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hw49s" event={"ID":"c4953609-37ed-4297-bb47-ddfc55ba2fb3","Type":"ContainerStarted","Data":"dd94f591a99319653cf0a6a46273423f6c4ab58dae0a58cdd86f802ccc3a8fd4"}
Jul 23 12:16:22.444879 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:22.444823    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mkjq8" event={"ID":"a0a054f4-3278-425f-ada5-65f0d5b116f2","Type":"ContainerStarted","Data":"bff919664bd750ddf8d5174c6d5eb5a3a8938adf25433b47fc0ca8c9870d9ea0"}
Jul 23 12:16:22.446243 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:22.446219    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hw49s" event={"ID":"c4953609-37ed-4297-bb47-ddfc55ba2fb3","Type":"ContainerStarted","Data":"8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780"}
Jul 23 12:16:22.446352 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:22.446276    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-hw49s" podUID="c4953609-37ed-4297-bb47-ddfc55ba2fb3" containerName="registry-server" containerID="cri-o://8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780" gracePeriod=2
Jul 23 12:16:22.462273 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:22.462228    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mkjq8" podStartSLOduration=2.024405689 podStartE2EDuration="3.462213474s" totalImagesPullingTime="1.437807785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:16:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:16:22.461827596 +0000 UTC m=+366.117336399" watchObservedRunningTime="2026-07-23 12:16:22.462213474 +0000 UTC m=+366.117722277"
Jul 23 12:16:22.482592 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:22.482542    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hw49s" podStartSLOduration=2.4644403759999998 podStartE2EDuration="4.482525456s" totalImagesPullingTime="2.01808508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:16:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:16:22.481244211 +0000 UTC m=+366.136753038" watchObservedRunningTime="2026-07-23 12:16:22.482525456 +0000 UTC m=+366.138034600"
Jul 23 12:16:22.694068 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:22.694042    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hw49s"
Jul 23 12:16:22.808094 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:22.808072    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh9cz\" (UniqueName: \"kubernetes.io/projected/c4953609-37ed-4297-bb47-ddfc55ba2fb3-kube-api-access-qh9cz\") pod \"c4953609-37ed-4297-bb47-ddfc55ba2fb3\" (UID: \"c4953609-37ed-4297-bb47-ddfc55ba2fb3\") "
Jul 23 12:16:22.810233 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:22.810205    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4953609-37ed-4297-bb47-ddfc55ba2fb3-kube-api-access-qh9cz" (OuterVolumeSpecName: "kube-api-access-qh9cz") pod "c4953609-37ed-4297-bb47-ddfc55ba2fb3" (UID: "c4953609-37ed-4297-bb47-ddfc55ba2fb3"). InnerVolumeSpecName "kube-api-access-qh9cz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:16:22.909050 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:22.909019    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh9cz\" (UniqueName: \"kubernetes.io/projected/c4953609-37ed-4297-bb47-ddfc55ba2fb3-kube-api-access-qh9cz\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:16:23.451063 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:23.451026    2565 generic.go:356] "Generic (PLEG): container finished" podID="c4953609-37ed-4297-bb47-ddfc55ba2fb3" containerID="8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780" exitCode=0
Jul 23 12:16:23.451063 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:23.451070    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hw49s"
Jul 23 12:16:23.451542 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:23.451111    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hw49s" event={"ID":"c4953609-37ed-4297-bb47-ddfc55ba2fb3","Type":"ContainerDied","Data":"8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780"}
Jul 23 12:16:23.451542 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:23.451161    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hw49s" event={"ID":"c4953609-37ed-4297-bb47-ddfc55ba2fb3","Type":"ContainerDied","Data":"dd94f591a99319653cf0a6a46273423f6c4ab58dae0a58cdd86f802ccc3a8fd4"}
Jul 23 12:16:23.451542 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:23.451185    2565 scope.go:117] "RemoveContainer" containerID="8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780"
Jul 23 12:16:23.459130 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:23.459094    2565 scope.go:117] "RemoveContainer" containerID="8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780"
Jul 23 12:16:23.459333 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:16:23.459317    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780\": container with ID starting with 8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780 not found: ID does not exist" containerID="8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780"
Jul 23 12:16:23.459373 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:23.459340    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780"} err="failed to get container status \"8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780\": rpc error: code = NotFound desc = could not find container \"8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780\": container with ID starting with 8d96d29a10e305d80cde7f552ff30e4bed85637fa98d16f85f4ffe322ac4e780 not found: ID does not exist"
Jul 23 12:16:23.472169 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:23.472143    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hw49s"]
Jul 23 12:16:23.477228 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:23.477209    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hw49s"]
Jul 23 12:16:25.049023 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:25.048994    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4953609-37ed-4297-bb47-ddfc55ba2fb3" path="/var/lib/kubelet/pods/c4953609-37ed-4297-bb47-ddfc55ba2fb3/volumes"
Jul 23 12:16:29.886392 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:29.886356    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-mkjq8"
Jul 23 12:16:29.886836 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:29.886412    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-mkjq8"
Jul 23 12:16:29.908590 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:29.908565    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-mkjq8"
Jul 23 12:16:30.500550 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:30.500521    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-mkjq8"
Jul 23 12:16:50.418816 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.418786    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"]
Jul 23 12:16:50.419283 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.419057    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4953609-37ed-4297-bb47-ddfc55ba2fb3" containerName="registry-server"
Jul 23 12:16:50.419283 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.419069    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4953609-37ed-4297-bb47-ddfc55ba2fb3" containerName="registry-server"
Jul 23 12:16:50.419283 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.419130    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4953609-37ed-4297-bb47-ddfc55ba2fb3" containerName="registry-server"
Jul 23 12:16:50.426443 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.426428    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"
Jul 23 12:16:50.429462 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.429445    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-nvrnr\""
Jul 23 12:16:50.429529 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.429467    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 23 12:16:50.437424 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.437401    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"]
Jul 23 12:16:50.513112 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.513077    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn9w9\" (UniqueName: \"kubernetes.io/projected/3f87b170-6812-4707-bd46-87bf942c00af-kube-api-access-mn9w9\") pod \"dns-operator-controller-manager-864dd78c4c-nc2t9\" (UID: \"3f87b170-6812-4707-bd46-87bf942c00af\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"
Jul 23 12:16:50.614328 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.614299    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn9w9\" (UniqueName: \"kubernetes.io/projected/3f87b170-6812-4707-bd46-87bf942c00af-kube-api-access-mn9w9\") pod \"dns-operator-controller-manager-864dd78c4c-nc2t9\" (UID: \"3f87b170-6812-4707-bd46-87bf942c00af\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"
Jul 23 12:16:50.624316 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.624279    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn9w9\" (UniqueName: \"kubernetes.io/projected/3f87b170-6812-4707-bd46-87bf942c00af-kube-api-access-mn9w9\") pod \"dns-operator-controller-manager-864dd78c4c-nc2t9\" (UID: \"3f87b170-6812-4707-bd46-87bf942c00af\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"
Jul 23 12:16:50.736896 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.736799    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"
Jul 23 12:16:50.867284 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:50.867190    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"]
Jul 23 12:16:50.869813 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:16:50.869788    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f87b170_6812_4707_bd46_87bf942c00af.slice/crio-8db9b67e10b050092ab6b57bf4bd22c050962c5e9558be30937276a41b761e5f WatchSource:0}: Error finding container 8db9b67e10b050092ab6b57bf4bd22c050962c5e9558be30937276a41b761e5f: Status 404 returned error can't find the container with id 8db9b67e10b050092ab6b57bf4bd22c050962c5e9558be30937276a41b761e5f
Jul 23 12:16:51.559435 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:51.559403    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9" event={"ID":"3f87b170-6812-4707-bd46-87bf942c00af","Type":"ContainerStarted","Data":"8db9b67e10b050092ab6b57bf4bd22c050962c5e9558be30937276a41b761e5f"}
Jul 23 12:16:53.569732 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:53.569698    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9" event={"ID":"3f87b170-6812-4707-bd46-87bf942c00af","Type":"ContainerStarted","Data":"0a1d4534cae7142575d2f53a171e7295d3c490bc260c0068c7641003939d2b40"}
Jul 23 12:16:53.570223 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:53.569831    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"
Jul 23 12:16:53.595955 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:53.595904    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9" podStartSLOduration=1.681279955 podStartE2EDuration="3.595887304s" totalImagesPullingTime="1.914607349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:16:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:16:53.59452854 +0000 UTC m=+397.250037345" watchObservedRunningTime="2026-07-23 12:16:53.595887304 +0000 UTC m=+397.251396107"
Jul 23 12:16:55.802162 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.802124    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"]
Jul 23 12:16:55.805665 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.805650    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:16:55.808606 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.808588    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5tb7f\""
Jul 23 12:16:55.820550 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.820529    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"]
Jul 23 12:16:55.855948 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.855915    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp2xp\" (UniqueName: \"kubernetes.io/projected/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-kube-api-access-kp2xp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" (UID: \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:16:55.856177 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.855966    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" (UID: \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:16:55.957361 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.957323    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2xp\" (UniqueName: \"kubernetes.io/projected/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-kube-api-access-kp2xp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" (UID: \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:16:55.957517 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.957379    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" (UID: \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:16:55.957756 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.957740    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" (UID: \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:16:55.968231 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:55.968208    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp2xp\" (UniqueName: \"kubernetes.io/projected/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-kube-api-access-kp2xp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" (UID: \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:16:56.115217 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:56.115165    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:16:56.256791 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:56.256735    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"]
Jul 23 12:16:56.257677 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:16:56.257652    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod692a8db8_a6fb_4fbb_bd2b_b77c14bcdf31.slice/crio-7c324bae25bacfc07668f9b3eee037911fc1e811aa67684e5a6d0efd336b46f2 WatchSource:0}: Error finding container 7c324bae25bacfc07668f9b3eee037911fc1e811aa67684e5a6d0efd336b46f2: Status 404 returned error can't find the container with id 7c324bae25bacfc07668f9b3eee037911fc1e811aa67684e5a6d0efd336b46f2
Jul 23 12:16:56.582217 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:56.582183    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w" event={"ID":"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31","Type":"ContainerStarted","Data":"7c324bae25bacfc07668f9b3eee037911fc1e811aa67684e5a6d0efd336b46f2"}
Jul 23 12:16:59.241305 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:59.241265    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"]
Jul 23 12:16:59.244798 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:59.244772    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"
Jul 23 12:16:59.248422 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:59.248381    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-c7288\""
Jul 23 12:16:59.281713 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:59.280981    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"]
Jul 23 12:16:59.284163 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:59.284089    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqwt7\" (UniqueName: \"kubernetes.io/projected/1d38df85-7e53-4ba1-bbf5-413590a41145-kube-api-access-wqwt7\") pod \"authorino-operator-75cc8d5cb7-8nrc9\" (UID: \"1d38df85-7e53-4ba1-bbf5-413590a41145\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"
Jul 23 12:16:59.384666 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:59.384598    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqwt7\" (UniqueName: \"kubernetes.io/projected/1d38df85-7e53-4ba1-bbf5-413590a41145-kube-api-access-wqwt7\") pod \"authorino-operator-75cc8d5cb7-8nrc9\" (UID: \"1d38df85-7e53-4ba1-bbf5-413590a41145\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"
Jul 23 12:16:59.401335 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:59.401309    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqwt7\" (UniqueName: \"kubernetes.io/projected/1d38df85-7e53-4ba1-bbf5-413590a41145-kube-api-access-wqwt7\") pod \"authorino-operator-75cc8d5cb7-8nrc9\" (UID: \"1d38df85-7e53-4ba1-bbf5-413590a41145\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"
Jul 23 12:16:59.558059 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:59.558029    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"
Jul 23 12:16:59.695881 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:16:59.695803    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"]
Jul 23 12:16:59.699126 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:16:59.699094    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d38df85_7e53_4ba1_bbf5_413590a41145.slice/crio-643e7b436584c4fa48e3c9f059a7f166b4fe4a9794106e32a8efbbda6809802e WatchSource:0}: Error finding container 643e7b436584c4fa48e3c9f059a7f166b4fe4a9794106e32a8efbbda6809802e: Status 404 returned error can't find the container with id 643e7b436584c4fa48e3c9f059a7f166b4fe4a9794106e32a8efbbda6809802e
Jul 23 12:17:00.601740 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:00.601701    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9" event={"ID":"1d38df85-7e53-4ba1-bbf5-413590a41145","Type":"ContainerStarted","Data":"643e7b436584c4fa48e3c9f059a7f166b4fe4a9794106e32a8efbbda6809802e"}
Jul 23 12:17:01.608323 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:01.608288    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w" event={"ID":"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31","Type":"ContainerStarted","Data":"97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db"}
Jul 23 12:17:01.608795 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:01.608430    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:01.633378 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:01.633319    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w" podStartSLOduration=1.8132537800000001 podStartE2EDuration="6.633301317s" totalImagesPullingTime="4.820047537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:17:01.630904352 +0000 UTC m=+405.286413157" watchObservedRunningTime="2026-07-23 12:17:01.633301317 +0000 UTC m=+405.288810121"
Jul 23 12:17:02.019837 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:02.019746    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"]
Jul 23 12:17:02.028458 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:02.028433    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:02.031717 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:02.031667    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ft8l4\""
Jul 23 12:17:02.033175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:02.033150    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"]
Jul 23 12:17:02.106798 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:02.106741    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffzxg\" (UniqueName: \"kubernetes.io/projected/22fa11e2-7e15-4569-a9b2-f56e34768868-kube-api-access-ffzxg\") pod \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" (UID: \"22fa11e2-7e15-4569-a9b2-f56e34768868\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:02.207895 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:02.207851    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffzxg\" (UniqueName: \"kubernetes.io/projected/22fa11e2-7e15-4569-a9b2-f56e34768868-kube-api-access-ffzxg\") pod \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" (UID: \"22fa11e2-7e15-4569-a9b2-f56e34768868\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:02.221667 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:02.221634    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffzxg\" (UniqueName: \"kubernetes.io/projected/22fa11e2-7e15-4569-a9b2-f56e34768868-kube-api-access-ffzxg\") pod \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" (UID: \"22fa11e2-7e15-4569-a9b2-f56e34768868\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:02.342437 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:02.342404    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:02.904192 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:02.904159    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"]
Jul 23 12:17:02.907147 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:17:02.907119    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22fa11e2_7e15_4569_a9b2_f56e34768868.slice/crio-d63801f562da94c3cb140a5394568bb9a2effd2a3a4a8c707b1ef00b89e63a16 WatchSource:0}: Error finding container d63801f562da94c3cb140a5394568bb9a2effd2a3a4a8c707b1ef00b89e63a16: Status 404 returned error can't find the container with id d63801f562da94c3cb140a5394568bb9a2effd2a3a4a8c707b1ef00b89e63a16
Jul 23 12:17:03.618827 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:03.618786    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9" event={"ID":"1d38df85-7e53-4ba1-bbf5-413590a41145","Type":"ContainerStarted","Data":"2915005b3abb26fc7e44b1beedaf59e14c685a9f8719caa3704da843d75ba707"}
Jul 23 12:17:03.619037 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:03.618860    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"
Jul 23 12:17:03.619984 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:03.619958    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh" event={"ID":"22fa11e2-7e15-4569-a9b2-f56e34768868","Type":"ContainerStarted","Data":"d63801f562da94c3cb140a5394568bb9a2effd2a3a4a8c707b1ef00b89e63a16"}
Jul 23 12:17:03.640011 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:03.639959    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9" podStartSLOduration=1.528269804 podStartE2EDuration="4.639939531s" totalImagesPullingTime="3.111669727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:17:03.637583051 +0000 UTC m=+407.293091854" watchObservedRunningTime="2026-07-23 12:17:03.639939531 +0000 UTC m=+407.295448335"
Jul 23 12:17:04.207736 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.207697    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"]
Jul 23 12:17:04.210998 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.210982    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.214013 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.213990    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 23 12:17:04.214121 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.214032    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-224vt\""
Jul 23 12:17:04.214121 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.214040    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 23 12:17:04.221448 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.221426    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"]
Jul 23 12:17:04.323736 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.323703    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49f2v\" (UniqueName: \"kubernetes.io/projected/426d23b7-7724-4b27-8b50-b8e81e46de5b-kube-api-access-49f2v\") pod \"kuadrant-console-plugin-66b5774985-xt2t5\" (UID: \"426d23b7-7724-4b27-8b50-b8e81e46de5b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.323928 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.323774    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/426d23b7-7724-4b27-8b50-b8e81e46de5b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xt2t5\" (UID: \"426d23b7-7724-4b27-8b50-b8e81e46de5b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.323928 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.323804    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/426d23b7-7724-4b27-8b50-b8e81e46de5b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xt2t5\" (UID: \"426d23b7-7724-4b27-8b50-b8e81e46de5b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.424184 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.424151    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49f2v\" (UniqueName: \"kubernetes.io/projected/426d23b7-7724-4b27-8b50-b8e81e46de5b-kube-api-access-49f2v\") pod \"kuadrant-console-plugin-66b5774985-xt2t5\" (UID: \"426d23b7-7724-4b27-8b50-b8e81e46de5b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.424346 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.424209    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/426d23b7-7724-4b27-8b50-b8e81e46de5b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xt2t5\" (UID: \"426d23b7-7724-4b27-8b50-b8e81e46de5b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.424346 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.424237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/426d23b7-7724-4b27-8b50-b8e81e46de5b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xt2t5\" (UID: \"426d23b7-7724-4b27-8b50-b8e81e46de5b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.424875 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.424857    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/426d23b7-7724-4b27-8b50-b8e81e46de5b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xt2t5\" (UID: \"426d23b7-7724-4b27-8b50-b8e81e46de5b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.426574 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.426558    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/426d23b7-7724-4b27-8b50-b8e81e46de5b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xt2t5\" (UID: \"426d23b7-7724-4b27-8b50-b8e81e46de5b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.437002 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.436979    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49f2v\" (UniqueName: \"kubernetes.io/projected/426d23b7-7724-4b27-8b50-b8e81e46de5b-kube-api-access-49f2v\") pod \"kuadrant-console-plugin-66b5774985-xt2t5\" (UID: \"426d23b7-7724-4b27-8b50-b8e81e46de5b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.521743 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.521661    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"
Jul 23 12:17:04.577158 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.577057    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nc2t9"
Jul 23 12:17:04.680519 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:04.680480    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5"]
Jul 23 12:17:04.682531 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:17:04.682500    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod426d23b7_7724_4b27_8b50_b8e81e46de5b.slice/crio-e2ae2f883929ba0eb65327a5055a53e2ca437fab2b8dc7e9af33a8201c6b37c1 WatchSource:0}: Error finding container e2ae2f883929ba0eb65327a5055a53e2ca437fab2b8dc7e9af33a8201c6b37c1: Status 404 returned error can't find the container with id e2ae2f883929ba0eb65327a5055a53e2ca437fab2b8dc7e9af33a8201c6b37c1
Jul 23 12:17:05.634341 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:05.634300    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh" event={"ID":"22fa11e2-7e15-4569-a9b2-f56e34768868","Type":"ContainerStarted","Data":"8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de"}
Jul 23 12:17:05.634796 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:05.634496    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:05.635631 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:05.635610    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5" event={"ID":"426d23b7-7724-4b27-8b50-b8e81e46de5b","Type":"ContainerStarted","Data":"e2ae2f883929ba0eb65327a5055a53e2ca437fab2b8dc7e9af33a8201c6b37c1"}
Jul 23 12:17:05.671206 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:05.671093    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh" podStartSLOduration=1.160845548 podStartE2EDuration="3.671069864s" totalImagesPullingTime="2.510224316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:17:05.670796056 +0000 UTC m=+409.326304861" watchObservedRunningTime="2026-07-23 12:17:05.671069864 +0000 UTC m=+409.326578673"
Jul 23 12:17:12.616067 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:12.616035    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:14.541189 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.541158    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"]
Jul 23 12:17:14.541668 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.541351    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" containerName="manager" containerID="cri-o://97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db" gracePeriod=2
Jul 23 12:17:14.544067 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.543991    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:14.546584 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.546518    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"]
Jul 23 12:17:14.559834 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.559732    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"]
Jul 23 12:17:14.560017 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.559964    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" containerName="manager" containerID="cri-o://8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de" gracePeriod=2
Jul 23 12:17:14.562068 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.562014    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:14.562533 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.562498    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:14.568151 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.568077    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"]
Jul 23 12:17:14.572107 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.572081    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"]
Jul 23 12:17:14.572462 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.572445    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" containerName="manager"
Jul 23 12:17:14.572583 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.572572    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" containerName="manager"
Jul 23 12:17:14.572666 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.572658    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" containerName="manager"
Jul 23 12:17:14.572726 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.572719    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" containerName="manager"
Jul 23 12:17:14.572900 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.572890    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" containerName="manager"
Jul 23 12:17:14.572978 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.572969    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" containerName="manager"
Jul 23 12:17:14.573374 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.573341    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:14.576635 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.576605    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:14.578302 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.578281    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:14.580867 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.580841    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:14.585337 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.585217    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:14.589656 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.589632    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"]
Jul 23 12:17:14.596030 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.594839    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"
Jul 23 12:17:14.597528 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.597500    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"]
Jul 23 12:17:14.614849 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.614820    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"]
Jul 23 12:17:14.618995 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.618868    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:14.621536 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.621508    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:14.629224 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.629201    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8nrc9"
Jul 23 12:17:14.644247 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.644210    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:14.669841 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:14.669801    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:14.718249 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.718209    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjcl\" (UniqueName: \"kubernetes.io/projected/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-kube-api-access-lsjcl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdp2g\" (UID: \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:14.718425 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.718315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcxv4\" (UniqueName: \"kubernetes.io/projected/313fa7b5-afb2-4756-b983-19ca5536c3c4-kube-api-access-jcxv4\") pod \"limitador-operator-controller-manager-859d7b78bd-dzvlb\" (UID: \"313fa7b5-afb2-4756-b983-19ca5536c3c4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"
Jul 23 12:17:14.718425 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.718353    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdp2g\" (UID: \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:14.818953 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.818915    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjcl\" (UniqueName: \"kubernetes.io/projected/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-kube-api-access-lsjcl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdp2g\" (UID: \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:14.819148 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.819003    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcxv4\" (UniqueName: \"kubernetes.io/projected/313fa7b5-afb2-4756-b983-19ca5536c3c4-kube-api-access-jcxv4\") pod \"limitador-operator-controller-manager-859d7b78bd-dzvlb\" (UID: \"313fa7b5-afb2-4756-b983-19ca5536c3c4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"
Jul 23 12:17:14.819148 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.819040    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdp2g\" (UID: \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:14.819456 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.819428    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdp2g\" (UID: \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:14.830469 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.830442    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjcl\" (UniqueName: \"kubernetes.io/projected/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-kube-api-access-lsjcl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdp2g\" (UID: \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:14.830592 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.830441    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcxv4\" (UniqueName: \"kubernetes.io/projected/313fa7b5-afb2-4756-b983-19ca5536c3c4-kube-api-access-jcxv4\") pod \"limitador-operator-controller-manager-859d7b78bd-dzvlb\" (UID: \"313fa7b5-afb2-4756-b983-19ca5536c3c4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"
Jul 23 12:17:14.957072 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.957035    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:14.964194 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:14.964158    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"
Jul 23 12:17:26.611668 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.611638    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:26.614523 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.614490    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:26.623269 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.623234    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"]
Jul 23 12:17:26.624429 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:17:26.624401    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccf57fbf_c7f3_49fb_ab96_085bd81ed0f0.slice/crio-785b0cc6ea358431495dd1b8262f1afa02985d441c7a79e2cba0b8700141ef02 WatchSource:0}: Error finding container 785b0cc6ea358431495dd1b8262f1afa02985d441c7a79e2cba0b8700141ef02: Status 404 returned error can't find the container with id 785b0cc6ea358431495dd1b8262f1afa02985d441c7a79e2cba0b8700141ef02
Jul 23 12:17:26.627383 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.627367    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:26.629957 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.629928    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:26.632341 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.632312    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:26.644941 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.644915    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"]
Jul 23 12:17:26.647501 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:17:26.647476    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313fa7b5_afb2_4756_b983_19ca5536c3c4.slice/crio-5c07142ac164f88a4989b7caa52d4cc6f0d8cf3e2b23e134b606da7d25ee4a02 WatchSource:0}: Error finding container 5c07142ac164f88a4989b7caa52d4cc6f0d8cf3e2b23e134b606da7d25ee4a02: Status 404 returned error can't find the container with id 5c07142ac164f88a4989b7caa52d4cc6f0d8cf3e2b23e134b606da7d25ee4a02
Jul 23 12:17:26.725168 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.725143    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-extensions-socket-volume\") pod \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\" (UID: \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\") "
Jul 23 12:17:26.725300 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.725211    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffzxg\" (UniqueName: \"kubernetes.io/projected/22fa11e2-7e15-4569-a9b2-f56e34768868-kube-api-access-ffzxg\") pod \"22fa11e2-7e15-4569-a9b2-f56e34768868\" (UID: \"22fa11e2-7e15-4569-a9b2-f56e34768868\") "
Jul 23 12:17:26.725300 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.725232    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp2xp\" (UniqueName: \"kubernetes.io/projected/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-kube-api-access-kp2xp\") pod \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\" (UID: \"692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31\") "
Jul 23 12:17:26.725607 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.725583    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" (UID: "692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:17:26.727191 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.727174    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-kube-api-access-kp2xp" (OuterVolumeSpecName: "kube-api-access-kp2xp") pod "692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" (UID: "692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31"). InnerVolumeSpecName "kube-api-access-kp2xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:17:26.727259 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.727207    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22fa11e2-7e15-4569-a9b2-f56e34768868-kube-api-access-ffzxg" (OuterVolumeSpecName: "kube-api-access-ffzxg") pod "22fa11e2-7e15-4569-a9b2-f56e34768868" (UID: "22fa11e2-7e15-4569-a9b2-f56e34768868"). InnerVolumeSpecName "kube-api-access-ffzxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:17:26.741733 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.741699    2565 generic.go:356] "Generic (PLEG): container finished" podID="22fa11e2-7e15-4569-a9b2-f56e34768868" containerID="8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de" exitCode=0
Jul 23 12:17:26.741893 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.741750    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:26.741893 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.741806    2565 scope.go:117] "RemoveContainer" containerID="8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de"
Jul 23 12:17:26.744132 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.744104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5" event={"ID":"426d23b7-7724-4b27-8b50-b8e81e46de5b","Type":"ContainerStarted","Data":"07dd772197de3495956fd58ade431cfdd8ea637bfdb9440bd0d26071c1cd8ce2"}
Jul 23 12:17:26.744711 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.744657    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:26.746346 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.746311    2565 generic.go:356] "Generic (PLEG): container finished" podID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" containerID="97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db" exitCode=0
Jul 23 12:17:26.746450 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.746365    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:26.747291 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.747253    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:26.748632 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.748605    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g" event={"ID":"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0","Type":"ContainerStarted","Data":"b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe"}
Jul 23 12:17:26.748727 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.748657    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g" event={"ID":"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0","Type":"ContainerStarted","Data":"785b0cc6ea358431495dd1b8262f1afa02985d441c7a79e2cba0b8700141ef02"}
Jul 23 12:17:26.748727 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.748687    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:26.750720 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.750607    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb" event={"ID":"313fa7b5-afb2-4756-b983-19ca5536c3c4","Type":"ContainerStarted","Data":"1de671ddf09334d59209cf3c11d19ca7716214cce1427f8d46e1ff39b93bfede"}
Jul 23 12:17:26.750720 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.750656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb" event={"ID":"313fa7b5-afb2-4756-b983-19ca5536c3c4","Type":"ContainerStarted","Data":"5c07142ac164f88a4989b7caa52d4cc6f0d8cf3e2b23e134b606da7d25ee4a02"}
Jul 23 12:17:26.751085 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.751060    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"
Jul 23 12:17:26.752693 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.752677    2565 scope.go:117] "RemoveContainer" containerID="8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de"
Jul 23 12:17:26.752960 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.752943    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de\": container with ID starting with 8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de not found: ID does not exist" containerID="8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de"
Jul 23 12:17:26.753013 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.752968    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de"} err="failed to get container status \"8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de\": rpc error: code = NotFound desc = could not find container \"8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de\": container with ID starting with 8daad2ea6514aee3d63e4010da0a932cf008b3ce8166c784ae883fa51e1a51de not found: ID does not exist"
Jul 23 12:17:26.753013 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.752984    2565 scope.go:117] "RemoveContainer" containerID="97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db"
Jul 23 12:17:26.760322 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.760308    2565 scope.go:117] "RemoveContainer" containerID="97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db"
Jul 23 12:17:26.760583 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.760559    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db\": container with ID starting with 97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db not found: ID does not exist" containerID="97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db"
Jul 23 12:17:26.760668 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.760583    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db"} err="failed to get container status \"97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db\": rpc error: code = NotFound desc = could not find container \"97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db\": container with ID starting with 97d97d9f6e61a5688149156e5e1d56dc71af119919d031af59be3788c18514db not found: ID does not exist"
Jul 23 12:17:26.766949 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.766913    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xt2t5" podStartSLOduration=0.900168075 podStartE2EDuration="22.766897355s" totalImagesPullingTime="21.86672928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:17:26.766706163 +0000 UTC m=+430.422214966" watchObservedRunningTime="2026-07-23 12:17:26.766897355 +0000 UTC m=+430.422406158"
Jul 23 12:17:26.771295 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.771272    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:26.773305 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.773285    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:26.795656 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.795610    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g" podStartSLOduration=12.795589656 podStartE2EDuration="12.795589656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:17:26.79497969 +0000 UTC m=+430.450488493" watchObservedRunningTime="2026-07-23 12:17:26.795589656 +0000 UTC m=+430.451098462"
Jul 23 12:17:26.814906 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.814863    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb" podStartSLOduration=12.814849441 podStartE2EDuration="12.814849441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:17:26.813686975 +0000 UTC m=+430.469195782" watchObservedRunningTime="2026-07-23 12:17:26.814849441 +0000 UTC m=+430.470358245"
Jul 23 12:17:26.815945 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.815920    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:26.818133 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:26.818113    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:26.826489 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.826468    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-extensions-socket-volume\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:17:26.826489 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.826487    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffzxg\" (UniqueName: \"kubernetes.io/projected/22fa11e2-7e15-4569-a9b2-f56e34768868-kube-api-access-ffzxg\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:17:26.826640 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:26.826498    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp2xp\" (UniqueName: \"kubernetes.io/projected/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31-kube-api-access-kp2xp\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:17:27.048071 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:27.047985    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6g5dh\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6g5dh"
Jul 23 12:17:27.049010 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:27.048988    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22fa11e2-7e15-4569-a9b2-f56e34768868" path="/var/lib/kubelet/pods/22fa11e2-7e15-4569-a9b2-f56e34768868/volumes"
Jul 23 12:17:27.049318 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:27.049306    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" path="/var/lib/kubelet/pods/692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31/volumes"
Jul 23 12:17:27.050468 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:27.050447    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv69w\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="692a8db8-a6fb-4fbb-bd2b-b77c14bcdf31" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv69w"
Jul 23 12:17:31.340160 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.340122    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"]
Jul 23 12:17:31.340633 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.340351    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g" podUID="ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0" containerName="manager" containerID="cri-o://b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe" gracePeriod=10
Jul 23 12:17:31.342265 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.342242    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:31.584303 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.584281    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:31.598476 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.598419    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"]
Jul 23 12:17:31.598706 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.598695    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0" containerName="manager"
Jul 23 12:17:31.598747 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.598708    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0" containerName="manager"
Jul 23 12:17:31.598802 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.598775    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0" containerName="manager"
Jul 23 12:17:31.601311 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.601292    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:17:31.614792 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.614752    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"]
Jul 23 12:17:31.663248 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.663209    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-extensions-socket-volume\") pod \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\" (UID: \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\") "
Jul 23 12:17:31.663421 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.663318    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f7z7\" (UniqueName: \"kubernetes.io/projected/917678ef-368e-4107-8534-66330a604db2-kube-api-access-5f7z7\") pod \"kuadrant-operator-controller-manager-86878c769f-xw8v4\" (UID: \"917678ef-368e-4107-8534-66330a604db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:17:31.663421 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.663377    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917678ef-368e-4107-8534-66330a604db2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xw8v4\" (UID: \"917678ef-368e-4107-8534-66330a604db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:17:31.663577 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.663558    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0" (UID: "ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:17:31.764464 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.764426    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsjcl\" (UniqueName: \"kubernetes.io/projected/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-kube-api-access-lsjcl\") pod \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\" (UID: \"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0\") "
Jul 23 12:17:31.764638 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.764618    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f7z7\" (UniqueName: \"kubernetes.io/projected/917678ef-368e-4107-8534-66330a604db2-kube-api-access-5f7z7\") pod \"kuadrant-operator-controller-manager-86878c769f-xw8v4\" (UID: \"917678ef-368e-4107-8534-66330a604db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:17:31.764743 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.764726    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917678ef-368e-4107-8534-66330a604db2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xw8v4\" (UID: \"917678ef-368e-4107-8534-66330a604db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:17:31.764852 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.764837    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-extensions-socket-volume\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:17:31.765107 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.765086    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917678ef-368e-4107-8534-66330a604db2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xw8v4\" (UID: \"917678ef-368e-4107-8534-66330a604db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:17:31.766666 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.766648    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-kube-api-access-lsjcl" (OuterVolumeSpecName: "kube-api-access-lsjcl") pod "ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0" (UID: "ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0"). InnerVolumeSpecName "kube-api-access-lsjcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:17:31.779464 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.779434    2565 generic.go:356] "Generic (PLEG): container finished" podID="ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0" containerID="b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe" exitCode=0
Jul 23 12:17:31.779585 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.779465    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g" event={"ID":"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0","Type":"ContainerDied","Data":"b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe"}
Jul 23 12:17:31.779585 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.779494    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"
Jul 23 12:17:31.779585 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.779446    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f7z7\" (UniqueName: \"kubernetes.io/projected/917678ef-368e-4107-8534-66330a604db2-kube-api-access-5f7z7\") pod \"kuadrant-operator-controller-manager-86878c769f-xw8v4\" (UID: \"917678ef-368e-4107-8534-66330a604db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:17:31.779585 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.779548    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g" event={"ID":"ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0","Type":"ContainerDied","Data":"785b0cc6ea358431495dd1b8262f1afa02985d441c7a79e2cba0b8700141ef02"}
Jul 23 12:17:31.779585 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.779563    2565 scope.go:117] "RemoveContainer" containerID="b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe"
Jul 23 12:17:31.790185 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.790159    2565 scope.go:117] "RemoveContainer" containerID="b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe"
Jul 23 12:17:31.790448 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:17:31.790429    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe\": container with ID starting with b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe not found: ID does not exist" containerID="b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe"
Jul 23 12:17:31.790505 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.790460    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe"} err="failed to get container status \"b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe\": rpc error: code = NotFound desc = could not find container \"b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe\": container with ID starting with b238ef5780e60f1717d1a66f19d085a4202cf958597c20f32b0d097e3c3d1dbe not found: ID does not exist"
Jul 23 12:17:31.812104 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.812077    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"]
Jul 23 12:17:31.818809 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.818785    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdp2g"]
Jul 23 12:17:31.865989 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.865915    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsjcl\" (UniqueName: \"kubernetes.io/projected/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0-kube-api-access-lsjcl\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:17:31.910875 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:31.910830    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:17:32.046917 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:32.046884    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"]
Jul 23 12:17:32.051005 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:17:32.050981    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod917678ef_368e_4107_8534_66330a604db2.slice/crio-1d98bd628118264e7f1c3f0b79aad4091cfac573b14f3e819ed050ec7d998f9b WatchSource:0}: Error finding container 1d98bd628118264e7f1c3f0b79aad4091cfac573b14f3e819ed050ec7d998f9b: Status 404 returned error can't find the container with id 1d98bd628118264e7f1c3f0b79aad4091cfac573b14f3e819ed050ec7d998f9b
Jul 23 12:17:32.786401 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:32.786368    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4" event={"ID":"917678ef-368e-4107-8534-66330a604db2","Type":"ContainerStarted","Data":"313cabf393e8eb9239a18c647952ff26ce0b9c9b860db5df1d9a1393de5700ae"}
Jul 23 12:17:32.786838 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:32.786417    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4" event={"ID":"917678ef-368e-4107-8534-66330a604db2","Type":"ContainerStarted","Data":"1d98bd628118264e7f1c3f0b79aad4091cfac573b14f3e819ed050ec7d998f9b"}
Jul 23 12:17:32.786838 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:32.786513    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:17:32.808319 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:32.808271    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4" podStartSLOduration=1.808254534 podStartE2EDuration="1.808254534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:17:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:17:32.807218259 +0000 UTC m=+436.462727074" watchObservedRunningTime="2026-07-23 12:17:32.808254534 +0000 UTC m=+436.463763337"
Jul 23 12:17:33.050073 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:33.049993    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0" path="/var/lib/kubelet/pods/ccf57fbf-c7f3-49fb-ab96-085bd81ed0f0/volumes"
Jul 23 12:17:38.763138 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:38.763068    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzvlb"
Jul 23 12:17:43.793175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:17:43.793149    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:18:01.626480 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.626444    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"]
Jul 23 12:18:01.653782 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.653727    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"]
Jul 23 12:18:01.653945 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.653857    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:01.656621 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.656599    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 12:18:01.702074 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.702043    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89618075-5beb-4cb7-bb6a-76ee8100e10c-config-file\") pod \"limitador-limitador-64c94fd64f-f2gx8\" (UID: \"89618075-5beb-4cb7-bb6a-76ee8100e10c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:01.702237 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.702093    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mc6r\" (UniqueName: \"kubernetes.io/projected/89618075-5beb-4cb7-bb6a-76ee8100e10c-kube-api-access-2mc6r\") pod \"limitador-limitador-64c94fd64f-f2gx8\" (UID: \"89618075-5beb-4cb7-bb6a-76ee8100e10c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:01.727638 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.727603    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"]
Jul 23 12:18:01.802784 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.802732    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89618075-5beb-4cb7-bb6a-76ee8100e10c-config-file\") pod \"limitador-limitador-64c94fd64f-f2gx8\" (UID: \"89618075-5beb-4cb7-bb6a-76ee8100e10c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:01.802952 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.802881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mc6r\" (UniqueName: \"kubernetes.io/projected/89618075-5beb-4cb7-bb6a-76ee8100e10c-kube-api-access-2mc6r\") pod \"limitador-limitador-64c94fd64f-f2gx8\" (UID: \"89618075-5beb-4cb7-bb6a-76ee8100e10c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:01.803398 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.803378    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89618075-5beb-4cb7-bb6a-76ee8100e10c-config-file\") pod \"limitador-limitador-64c94fd64f-f2gx8\" (UID: \"89618075-5beb-4cb7-bb6a-76ee8100e10c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:01.815893 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.815859    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mc6r\" (UniqueName: \"kubernetes.io/projected/89618075-5beb-4cb7-bb6a-76ee8100e10c-kube-api-access-2mc6r\") pod \"limitador-limitador-64c94fd64f-f2gx8\" (UID: \"89618075-5beb-4cb7-bb6a-76ee8100e10c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:01.964003 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:01.963915    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:02.112197 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:02.112097    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"]
Jul 23 12:18:02.114891 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:18:02.114861    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89618075_5beb_4cb7_bb6a_76ee8100e10c.slice/crio-bde8e9aa685f448d9ddf08d17fe0590a18c7aadbb8585e9c53b965c5fd0dd452 WatchSource:0}: Error finding container bde8e9aa685f448d9ddf08d17fe0590a18c7aadbb8585e9c53b965c5fd0dd452: Status 404 returned error can't find the container with id bde8e9aa685f448d9ddf08d17fe0590a18c7aadbb8585e9c53b965c5fd0dd452
Jul 23 12:18:02.918822 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:02.918791    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8" event={"ID":"89618075-5beb-4cb7-bb6a-76ee8100e10c","Type":"ContainerStarted","Data":"bde8e9aa685f448d9ddf08d17fe0590a18c7aadbb8585e9c53b965c5fd0dd452"}
Jul 23 12:18:05.935596 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:05.935562    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8" event={"ID":"89618075-5beb-4cb7-bb6a-76ee8100e10c","Type":"ContainerStarted","Data":"41778dcedf829019cf66921f92e6ca5deba584f1eb38780ffd976f5899ede4e5"}
Jul 23 12:18:05.935988 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:05.935683    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:05.955616 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:05.955568    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8" podStartSLOduration=2.129278787 podStartE2EDuration="4.955550666s" totalImagesPullingTime="2.826271879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:18:05.955163452 +0000 UTC m=+469.610672258" watchObservedRunningTime="2026-07-23 12:18:05.955550666 +0000 UTC m=+469.611059471"
Jul 23 12:18:16.554210 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:16.554164    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"]
Jul 23 12:18:16.554832 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:16.554393    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8" podUID="89618075-5beb-4cb7-bb6a-76ee8100e10c" containerName="limitador" containerID="cri-o://41778dcedf829019cf66921f92e6ca5deba584f1eb38780ffd976f5899ede4e5" gracePeriod=30
Jul 23 12:18:16.555204 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:16.555049    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:16.983837 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:16.983805    2565 generic.go:356] "Generic (PLEG): container finished" podID="89618075-5beb-4cb7-bb6a-76ee8100e10c" containerID="41778dcedf829019cf66921f92e6ca5deba584f1eb38780ffd976f5899ede4e5" exitCode=0
Jul 23 12:18:16.983986 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:16.983877    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8" event={"ID":"89618075-5beb-4cb7-bb6a-76ee8100e10c","Type":"ContainerDied","Data":"41778dcedf829019cf66921f92e6ca5deba584f1eb38780ffd976f5899ede4e5"}
Jul 23 12:18:17.492096 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.492072    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:17.643684 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.643575    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mc6r\" (UniqueName: \"kubernetes.io/projected/89618075-5beb-4cb7-bb6a-76ee8100e10c-kube-api-access-2mc6r\") pod \"89618075-5beb-4cb7-bb6a-76ee8100e10c\" (UID: \"89618075-5beb-4cb7-bb6a-76ee8100e10c\") "
Jul 23 12:18:17.644054 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.643687    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89618075-5beb-4cb7-bb6a-76ee8100e10c-config-file\") pod \"89618075-5beb-4cb7-bb6a-76ee8100e10c\" (UID: \"89618075-5beb-4cb7-bb6a-76ee8100e10c\") "
Jul 23 12:18:17.644054 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.644034    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89618075-5beb-4cb7-bb6a-76ee8100e10c-config-file" (OuterVolumeSpecName: "config-file") pod "89618075-5beb-4cb7-bb6a-76ee8100e10c" (UID: "89618075-5beb-4cb7-bb6a-76ee8100e10c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:18:17.645660 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.645637    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89618075-5beb-4cb7-bb6a-76ee8100e10c-kube-api-access-2mc6r" (OuterVolumeSpecName: "kube-api-access-2mc6r") pod "89618075-5beb-4cb7-bb6a-76ee8100e10c" (UID: "89618075-5beb-4cb7-bb6a-76ee8100e10c"). InnerVolumeSpecName "kube-api-access-2mc6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:18:17.744245 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.744208    2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89618075-5beb-4cb7-bb6a-76ee8100e10c-config-file\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:18:17.744245 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.744239    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mc6r\" (UniqueName: \"kubernetes.io/projected/89618075-5beb-4cb7-bb6a-76ee8100e10c-kube-api-access-2mc6r\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:18:17.989644 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.989563    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"
Jul 23 12:18:17.989815 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.989568    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f2gx8" event={"ID":"89618075-5beb-4cb7-bb6a-76ee8100e10c","Type":"ContainerDied","Data":"bde8e9aa685f448d9ddf08d17fe0590a18c7aadbb8585e9c53b965c5fd0dd452"}
Jul 23 12:18:17.989815 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:17.989700    2565 scope.go:117] "RemoveContainer" containerID="41778dcedf829019cf66921f92e6ca5deba584f1eb38780ffd976f5899ede4e5"
Jul 23 12:18:18.014996 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:18.014975    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"]
Jul 23 12:18:18.019276 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:18.019256    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f2gx8"]
Jul 23 12:18:19.049174 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:19.049147    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89618075-5beb-4cb7-bb6a-76ee8100e10c" path="/var/lib/kubelet/pods/89618075-5beb-4cb7-bb6a-76ee8100e10c/volumes"
Jul 23 12:18:23.431201 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.431170    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"]
Jul 23 12:18:23.431626 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.431612    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89618075-5beb-4cb7-bb6a-76ee8100e10c" containerName="limitador"
Jul 23 12:18:23.431680 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.431630    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="89618075-5beb-4cb7-bb6a-76ee8100e10c" containerName="limitador"
Jul 23 12:18:23.431731 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.431719    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="89618075-5beb-4cb7-bb6a-76ee8100e10c" containerName="limitador"
Jul 23 12:18:23.434885 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.434866    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:23.437859 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.437835    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 12:18:23.437859 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.437854    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 23 12:18:23.438035 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.437835    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 12:18:23.439008 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.438990    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-6dfjg\""
Jul 23 12:18:23.441847 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.441823    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"]
Jul 23 12:18:23.589451 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.589410    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf5q8\" (UniqueName: \"kubernetes.io/projected/2169ce02-42fb-4e1f-820d-0271b4086952-kube-api-access-sf5q8\") pod \"postgres-69fb4594cf-fxlxl\" (UID: \"2169ce02-42fb-4e1f-820d-0271b4086952\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:23.589641 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.589468    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2169ce02-42fb-4e1f-820d-0271b4086952-data\") pod \"postgres-69fb4594cf-fxlxl\" (UID: \"2169ce02-42fb-4e1f-820d-0271b4086952\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:23.690691 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.690601    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2169ce02-42fb-4e1f-820d-0271b4086952-data\") pod \"postgres-69fb4594cf-fxlxl\" (UID: \"2169ce02-42fb-4e1f-820d-0271b4086952\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:23.690691 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.690673    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf5q8\" (UniqueName: \"kubernetes.io/projected/2169ce02-42fb-4e1f-820d-0271b4086952-kube-api-access-sf5q8\") pod \"postgres-69fb4594cf-fxlxl\" (UID: \"2169ce02-42fb-4e1f-820d-0271b4086952\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:23.691007 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.690988    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2169ce02-42fb-4e1f-820d-0271b4086952-data\") pod \"postgres-69fb4594cf-fxlxl\" (UID: \"2169ce02-42fb-4e1f-820d-0271b4086952\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:23.710799 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.710754    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf5q8\" (UniqueName: \"kubernetes.io/projected/2169ce02-42fb-4e1f-820d-0271b4086952-kube-api-access-sf5q8\") pod \"postgres-69fb4594cf-fxlxl\" (UID: \"2169ce02-42fb-4e1f-820d-0271b4086952\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:23.746665 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.746634    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:23.877630 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:23.877597    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"]
Jul 23 12:18:24.017475 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:24.017392    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl" event={"ID":"2169ce02-42fb-4e1f-820d-0271b4086952","Type":"ContainerStarted","Data":"d66f13d2893d6a0731655f759ae03d69043caabca5043fc253b4e3a13719cd66"}
Jul 23 12:18:30.050034 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:30.049995    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl" event={"ID":"2169ce02-42fb-4e1f-820d-0271b4086952","Type":"ContainerStarted","Data":"1a756949e5c0c75cce3fc2ac59dd2b9198c6a1c31b9d338d6cfae8f39726f206"}
Jul 23 12:18:30.050480 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:30.050051    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:30.072365 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:30.072314    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl" podStartSLOduration=1.46007391 podStartE2EDuration="7.072298509s" totalImagesPullingTime="5.612224599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:18:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:18:30.071657137 +0000 UTC m=+493.727165942" watchObservedRunningTime="2026-07-23 12:18:30.072298509 +0000 UTC m=+493.727807313"
Jul 23 12:18:36.083993 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.083962    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fxlxl"
Jul 23 12:18:36.682364 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.682329    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j2jcz"]
Jul 23 12:18:36.688201 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.688174    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j2jcz"
Jul 23 12:18:36.691836 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.691811    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbgvz\" (UniqueName: \"kubernetes.io/projected/cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad-kube-api-access-cbgvz\") pod \"authorino-5d4b67c446-j2jcz\" (UID: \"cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad\") " pod="kuadrant-system/authorino-5d4b67c446-j2jcz"
Jul 23 12:18:36.693166 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.693141    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j2jcz"]
Jul 23 12:18:36.694226 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.694200    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tn957\""
Jul 23 12:18:36.792959 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.792931    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbgvz\" (UniqueName: \"kubernetes.io/projected/cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad-kube-api-access-cbgvz\") pod \"authorino-5d4b67c446-j2jcz\" (UID: \"cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad\") " pod="kuadrant-system/authorino-5d4b67c446-j2jcz"
Jul 23 12:18:36.802588 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.802557    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbgvz\" (UniqueName: \"kubernetes.io/projected/cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad-kube-api-access-cbgvz\") pod \"authorino-5d4b67c446-j2jcz\" (UID: \"cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad\") " pod="kuadrant-system/authorino-5d4b67c446-j2jcz"
Jul 23 12:18:36.921544 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.921511    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j2jcz"]
Jul 23 12:18:36.921753 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:36.921739    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j2jcz"
Jul 23 12:18:37.057213 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.057150    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j2jcz"]
Jul 23 12:18:37.059484 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:18:37.059460    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdcb5bc9_f108_4eaf_8a43_2daac1b0c5ad.slice/crio-f969299a0dd0b67a46836bfb6bccc0d4f10319217b3c8a3d2fae9935d9633922 WatchSource:0}: Error finding container f969299a0dd0b67a46836bfb6bccc0d4f10319217b3c8a3d2fae9935d9633922: Status 404 returned error can't find the container with id f969299a0dd0b67a46836bfb6bccc0d4f10319217b3c8a3d2fae9935d9633922
Jul 23 12:18:37.081175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.081149    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j2jcz" event={"ID":"cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad","Type":"ContainerStarted","Data":"f969299a0dd0b67a46836bfb6bccc0d4f10319217b3c8a3d2fae9935d9633922"}
Jul 23 12:18:37.427155 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.427119    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d79685df6-kx9mx"]
Jul 23 12:18:37.430087 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.430071    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:18:37.435119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.435093    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 23 12:18:37.443138 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.443108    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d79685df6-kx9mx"]
Jul 23 12:18:37.498890 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.498853    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmdcm\" (UniqueName: \"kubernetes.io/projected/3934214a-abb1-45f6-be0f-0e81f98c73a5-kube-api-access-nmdcm\") pod \"authorino-d79685df6-kx9mx\" (UID: \"3934214a-abb1-45f6-be0f-0e81f98c73a5\") " pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:18:37.499058 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.498908    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3934214a-abb1-45f6-be0f-0e81f98c73a5-tls-cert\") pod \"authorino-d79685df6-kx9mx\" (UID: \"3934214a-abb1-45f6-be0f-0e81f98c73a5\") " pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:18:37.599703 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.599666    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmdcm\" (UniqueName: \"kubernetes.io/projected/3934214a-abb1-45f6-be0f-0e81f98c73a5-kube-api-access-nmdcm\") pod \"authorino-d79685df6-kx9mx\" (UID: \"3934214a-abb1-45f6-be0f-0e81f98c73a5\") " pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:18:37.599886 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.599739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3934214a-abb1-45f6-be0f-0e81f98c73a5-tls-cert\") pod \"authorino-d79685df6-kx9mx\" (UID: \"3934214a-abb1-45f6-be0f-0e81f98c73a5\") " pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:18:37.602116 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.602096    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3934214a-abb1-45f6-be0f-0e81f98c73a5-tls-cert\") pod \"authorino-d79685df6-kx9mx\" (UID: \"3934214a-abb1-45f6-be0f-0e81f98c73a5\") " pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:18:37.612399 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.612375    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmdcm\" (UniqueName: \"kubernetes.io/projected/3934214a-abb1-45f6-be0f-0e81f98c73a5-kube-api-access-nmdcm\") pod \"authorino-d79685df6-kx9mx\" (UID: \"3934214a-abb1-45f6-be0f-0e81f98c73a5\") " pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:18:37.739471 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.739380    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:18:37.901425 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:37.901193    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d79685df6-kx9mx"]
Jul 23 12:18:37.904255 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:18:37.904225    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3934214a_abb1_45f6_be0f_0e81f98c73a5.slice/crio-a19f25f1e51075ea01acb2dfdba236b7c6df2239366a582aa7a38af6869e0cf0 WatchSource:0}: Error finding container a19f25f1e51075ea01acb2dfdba236b7c6df2239366a582aa7a38af6869e0cf0: Status 404 returned error can't find the container with id a19f25f1e51075ea01acb2dfdba236b7c6df2239366a582aa7a38af6869e0cf0
Jul 23 12:18:38.090280 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:38.090244    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d79685df6-kx9mx" event={"ID":"3934214a-abb1-45f6-be0f-0e81f98c73a5","Type":"ContainerStarted","Data":"a19f25f1e51075ea01acb2dfdba236b7c6df2239366a582aa7a38af6869e0cf0"}
Jul 23 12:18:41.107753 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:41.107715    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d79685df6-kx9mx" event={"ID":"3934214a-abb1-45f6-be0f-0e81f98c73a5","Type":"ContainerStarted","Data":"b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2"}
Jul 23 12:18:41.109069 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:41.109047    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j2jcz" event={"ID":"cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad","Type":"ContainerStarted","Data":"2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b"}
Jul 23 12:18:41.109225 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:41.109101    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-j2jcz" podUID="cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad" containerName="authorino" containerID="cri-o://2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b" gracePeriod=30
Jul 23 12:18:41.128060 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:41.128018    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d79685df6-kx9mx" podStartSLOduration=1.45237731 podStartE2EDuration="4.128004204s" totalImagesPullingTime="2.675626894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:18:41.127370727 +0000 UTC m=+504.782879543" watchObservedRunningTime="2026-07-23 12:18:41.128004204 +0000 UTC m=+504.783513007"
Jul 23 12:18:41.144609 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:41.144562    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-j2jcz" podStartSLOduration=1.633075475 podStartE2EDuration="5.144539557s" totalImagesPullingTime="3.511464082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:18:41.143626893 +0000 UTC m=+504.799135698" watchObservedRunningTime="2026-07-23 12:18:41.144539557 +0000 UTC m=+504.800048361"
Jul 23 12:18:41.353141 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:41.353117    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j2jcz"
Jul 23 12:18:41.429189 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:41.429107    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbgvz\" (UniqueName: \"kubernetes.io/projected/cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad-kube-api-access-cbgvz\") pod \"cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad\" (UID: \"cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad\") "
Jul 23 12:18:41.431119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:41.431089    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad-kube-api-access-cbgvz" (OuterVolumeSpecName: "kube-api-access-cbgvz") pod "cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad" (UID: "cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad"). InnerVolumeSpecName "kube-api-access-cbgvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:18:41.529979 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:41.529944    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbgvz\" (UniqueName: \"kubernetes.io/projected/cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad-kube-api-access-cbgvz\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:18:42.114636 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:42.114602    2565 generic.go:356] "Generic (PLEG): container finished" podID="cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad" containerID="2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b" exitCode=0
Jul 23 12:18:42.115055 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:42.114643    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j2jcz"
Jul 23 12:18:42.115055 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:42.114688    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j2jcz" event={"ID":"cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad","Type":"ContainerDied","Data":"2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b"}
Jul 23 12:18:42.115055 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:42.114724    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j2jcz" event={"ID":"cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad","Type":"ContainerDied","Data":"f969299a0dd0b67a46836bfb6bccc0d4f10319217b3c8a3d2fae9935d9633922"}
Jul 23 12:18:42.115055 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:42.114741    2565 scope.go:117] "RemoveContainer" containerID="2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b"
Jul 23 12:18:42.123213 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:42.123195    2565 scope.go:117] "RemoveContainer" containerID="2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b"
Jul 23 12:18:42.123450 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:18:42.123427    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b\": container with ID starting with 2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b not found: ID does not exist" containerID="2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b"
Jul 23 12:18:42.123500 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:42.123459    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b"} err="failed to get container status \"2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b\": rpc error: code = NotFound desc = could not find container \"2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b\": container with ID starting with 2f0ce08ce408f5c84db54d70b9bdda986827f9eca66dd0a721d9d8a29dead34b not found: ID does not exist"
Jul 23 12:18:42.143368 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:42.143337    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j2jcz"]
Jul 23 12:18:42.151386 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:42.151353    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j2jcz"]
Jul 23 12:18:43.049222 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:43.049189    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad" path="/var/lib/kubelet/pods/cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad/volumes"
Jul 23 12:18:46.030291 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.030257    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6db7d96575-wc445"]
Jul 23 12:18:46.030685 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.030664    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad" containerName="authorino"
Jul 23 12:18:46.030685 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.030680    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad" containerName="authorino"
Jul 23 12:18:46.030815 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.030744    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdcb5bc9-f108-4eaf-8a43-2daac1b0c5ad" containerName="authorino"
Jul 23 12:18:46.034010 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.033995    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:18:46.037233 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.037214    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9rcs9\""
Jul 23 12:18:46.037450 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.037437    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 12:18:46.038480 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.038466    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 12:18:46.045841 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.045815    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6db7d96575-wc445"]
Jul 23 12:18:46.163104 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.163077    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-webhook-cert\") pod \"maas-controller-6db7d96575-wc445\" (UID: \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\") " pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:18:46.163279 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.163125    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h27dw\" (UniqueName: \"kubernetes.io/projected/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-kube-api-access-h27dw\") pod \"maas-controller-6db7d96575-wc445\" (UID: \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\") " pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:18:46.214602 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.214570    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-645984f55d-wfs9p"]
Jul 23 12:18:46.216703 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.216686    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:18:46.229099 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.229074    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645984f55d-wfs9p"]
Jul 23 12:18:46.263563 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.263532    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0b675bc-c4c1-49ae-94ac-25149d61d33f-webhook-cert\") pod \"maas-controller-645984f55d-wfs9p\" (UID: \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\") " pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:18:46.263737 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.263574    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-webhook-cert\") pod \"maas-controller-6db7d96575-wc445\" (UID: \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\") " pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:18:46.263737 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.263591    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft6jn\" (UniqueName: \"kubernetes.io/projected/d0b675bc-c4c1-49ae-94ac-25149d61d33f-kube-api-access-ft6jn\") pod \"maas-controller-645984f55d-wfs9p\" (UID: \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\") " pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:18:46.263737 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.263667    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h27dw\" (UniqueName: \"kubernetes.io/projected/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-kube-api-access-h27dw\") pod \"maas-controller-6db7d96575-wc445\" (UID: \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\") " pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:18:46.265972 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.265943    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-webhook-cert\") pod \"maas-controller-6db7d96575-wc445\" (UID: \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\") " pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:18:46.273176 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.273155    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h27dw\" (UniqueName: \"kubernetes.io/projected/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-kube-api-access-h27dw\") pod \"maas-controller-6db7d96575-wc445\" (UID: \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\") " pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:18:46.344586 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.344561    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:18:46.364092 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.364060    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0b675bc-c4c1-49ae-94ac-25149d61d33f-webhook-cert\") pod \"maas-controller-645984f55d-wfs9p\" (UID: \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\") " pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:18:46.364213 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.364112    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft6jn\" (UniqueName: \"kubernetes.io/projected/d0b675bc-c4c1-49ae-94ac-25149d61d33f-kube-api-access-ft6jn\") pod \"maas-controller-645984f55d-wfs9p\" (UID: \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\") " pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:18:46.366478 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.366454    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0b675bc-c4c1-49ae-94ac-25149d61d33f-webhook-cert\") pod \"maas-controller-645984f55d-wfs9p\" (UID: \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\") " pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:18:46.374020 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.373998    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft6jn\" (UniqueName: \"kubernetes.io/projected/d0b675bc-c4c1-49ae-94ac-25149d61d33f-kube-api-access-ft6jn\") pod \"maas-controller-645984f55d-wfs9p\" (UID: \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\") " pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:18:46.478641 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.478485    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6db7d96575-wc445"]
Jul 23 12:18:46.480793 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:18:46.480753    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab7f85d3_f7fc_405c_b45e_2170ab877e3f.slice/crio-56f3007dfefdd2b899345acf1dab758274b0a9e64b993a048cf64d77349c2225 WatchSource:0}: Error finding container 56f3007dfefdd2b899345acf1dab758274b0a9e64b993a048cf64d77349c2225: Status 404 returned error can't find the container with id 56f3007dfefdd2b899345acf1dab758274b0a9e64b993a048cf64d77349c2225
Jul 23 12:18:46.526001 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.525971    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:18:46.660900 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:46.660866    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645984f55d-wfs9p"]
Jul 23 12:18:46.663614 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:18:46.663594    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b675bc_c4c1_49ae_94ac_25149d61d33f.slice/crio-ae49401561c663e34d6c76f3bf5861826261041bc0552eacba84aa278dd1a86f WatchSource:0}: Error finding container ae49401561c663e34d6c76f3bf5861826261041bc0552eacba84aa278dd1a86f: Status 404 returned error can't find the container with id ae49401561c663e34d6c76f3bf5861826261041bc0552eacba84aa278dd1a86f
Jul 23 12:18:47.142357 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:47.142314    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-wfs9p" event={"ID":"d0b675bc-c4c1-49ae-94ac-25149d61d33f","Type":"ContainerStarted","Data":"ae49401561c663e34d6c76f3bf5861826261041bc0552eacba84aa278dd1a86f"}
Jul 23 12:18:47.143316 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:47.143290    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6db7d96575-wc445" event={"ID":"ab7f85d3-f7fc-405c-b45e-2170ab877e3f","Type":"ContainerStarted","Data":"56f3007dfefdd2b899345acf1dab758274b0a9e64b993a048cf64d77349c2225"}
Jul 23 12:18:50.165430 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:50.165387    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-wfs9p" event={"ID":"d0b675bc-c4c1-49ae-94ac-25149d61d33f","Type":"ContainerStarted","Data":"439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b"}
Jul 23 12:18:50.165905 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:50.165517    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:18:50.166935 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:50.166912    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6db7d96575-wc445" event={"ID":"ab7f85d3-f7fc-405c-b45e-2170ab877e3f","Type":"ContainerStarted","Data":"52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71"}
Jul 23 12:18:50.167061 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:50.167036    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:18:50.185792 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:50.185727    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-645984f55d-wfs9p" podStartSLOduration=1.75377735 podStartE2EDuration="4.185710753s" totalImagesPullingTime="2.431933403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:18:50.183576149 +0000 UTC m=+513.839084956" watchObservedRunningTime="2026-07-23 12:18:50.185710753 +0000 UTC m=+513.841219575"
Jul 23 12:18:50.205080 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:18:50.205025    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6db7d96575-wc445" podStartSLOduration=1.5992687380000001 podStartE2EDuration="4.2050101s" totalImagesPullingTime="2.605741362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:18:50.204321531 +0000 UTC m=+513.859830368" watchObservedRunningTime="2026-07-23 12:18:50.2050101 +0000 UTC m=+513.860518907"
Jul 23 12:19:01.173700 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.173670    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:19:01.174081 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.173802    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:19:01.260362 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.260329    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6db7d96575-wc445"]
Jul 23 12:19:01.260571 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.260549    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6db7d96575-wc445" podUID="ab7f85d3-f7fc-405c-b45e-2170ab877e3f" containerName="manager" containerID="cri-o://52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71" gracePeriod=30
Jul 23 12:19:01.504358 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.504332    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:19:01.582651 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.582614    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-webhook-cert\") pod \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\" (UID: \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\") "
Jul 23 12:19:01.582826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.582715    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h27dw\" (UniqueName: \"kubernetes.io/projected/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-kube-api-access-h27dw\") pod \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\" (UID: \"ab7f85d3-f7fc-405c-b45e-2170ab877e3f\") "
Jul 23 12:19:01.584962 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.584924    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ab7f85d3-f7fc-405c-b45e-2170ab877e3f" (UID: "ab7f85d3-f7fc-405c-b45e-2170ab877e3f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:19:01.584962 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.584945    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-kube-api-access-h27dw" (OuterVolumeSpecName: "kube-api-access-h27dw") pod "ab7f85d3-f7fc-405c-b45e-2170ab877e3f" (UID: "ab7f85d3-f7fc-405c-b45e-2170ab877e3f"). InnerVolumeSpecName "kube-api-access-h27dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:19:01.683839 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.683725    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-webhook-cert\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:19:01.683839 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:01.683785    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h27dw\" (UniqueName: \"kubernetes.io/projected/ab7f85d3-f7fc-405c-b45e-2170ab877e3f-kube-api-access-h27dw\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:19:02.219771 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:02.219732    2565 generic.go:356] "Generic (PLEG): container finished" podID="ab7f85d3-f7fc-405c-b45e-2170ab877e3f" containerID="52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71" exitCode=0
Jul 23 12:19:02.220162 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:02.219789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6db7d96575-wc445" event={"ID":"ab7f85d3-f7fc-405c-b45e-2170ab877e3f","Type":"ContainerDied","Data":"52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71"}
Jul 23 12:19:02.220162 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:02.219799    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6db7d96575-wc445"
Jul 23 12:19:02.220162 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:02.219841    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6db7d96575-wc445" event={"ID":"ab7f85d3-f7fc-405c-b45e-2170ab877e3f","Type":"ContainerDied","Data":"56f3007dfefdd2b899345acf1dab758274b0a9e64b993a048cf64d77349c2225"}
Jul 23 12:19:02.220162 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:02.219870    2565 scope.go:117] "RemoveContainer" containerID="52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71"
Jul 23 12:19:02.228702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:02.228682    2565 scope.go:117] "RemoveContainer" containerID="52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71"
Jul 23 12:19:02.229014 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:19:02.228994    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71\": container with ID starting with 52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71 not found: ID does not exist" containerID="52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71"
Jul 23 12:19:02.229077 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:02.229023    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71"} err="failed to get container status \"52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71\": rpc error: code = NotFound desc = could not find container \"52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71\": container with ID starting with 52124eb8b6159f3a4573fcc95c73b8b83ec5ac88555e872c51d7df413d57dd71 not found: ID does not exist"
Jul 23 12:19:02.246006 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:02.245974    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6db7d96575-wc445"]
Jul 23 12:19:02.250050 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:02.250024    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6db7d96575-wc445"]
Jul 23 12:19:03.050035 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:03.050004    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab7f85d3-f7fc-405c-b45e-2170ab877e3f" path="/var/lib/kubelet/pods/ab7f85d3-f7fc-405c-b45e-2170ab877e3f/volumes"
Jul 23 12:19:24.108234 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.108153    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"]
Jul 23 12:19:24.108713 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.108484    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab7f85d3-f7fc-405c-b45e-2170ab877e3f" containerName="manager"
Jul 23 12:19:24.108713 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.108495    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab7f85d3-f7fc-405c-b45e-2170ab877e3f" containerName="manager"
Jul 23 12:19:24.108713 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.108572    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab7f85d3-f7fc-405c-b45e-2170ab877e3f" containerName="manager"
Jul 23 12:19:24.115315 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.115297    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.118423 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.118396    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 23 12:19:24.118529 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.118399    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fh87z\""
Jul 23 12:19:24.123063 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.123041    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"]
Jul 23 12:19:24.272645 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.272616    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5476f4d-df53-40ad-95d1-c737e8f209fa-maas-api-tls\") pod \"maas-api-776cd7fd75-gtnpj\" (UID: \"a5476f4d-df53-40ad-95d1-c737e8f209fa\") " pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.272834 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.272670    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5476f4d-df53-40ad-95d1-c737e8f209fa-openshift-service-ca\") pod \"maas-api-776cd7fd75-gtnpj\" (UID: \"a5476f4d-df53-40ad-95d1-c737e8f209fa\") " pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.272834 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.272719    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92bdm\" (UniqueName: \"kubernetes.io/projected/a5476f4d-df53-40ad-95d1-c737e8f209fa-kube-api-access-92bdm\") pod \"maas-api-776cd7fd75-gtnpj\" (UID: \"a5476f4d-df53-40ad-95d1-c737e8f209fa\") " pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.373665 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.373582    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92bdm\" (UniqueName: \"kubernetes.io/projected/a5476f4d-df53-40ad-95d1-c737e8f209fa-kube-api-access-92bdm\") pod \"maas-api-776cd7fd75-gtnpj\" (UID: \"a5476f4d-df53-40ad-95d1-c737e8f209fa\") " pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.373665 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.373635    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5476f4d-df53-40ad-95d1-c737e8f209fa-maas-api-tls\") pod \"maas-api-776cd7fd75-gtnpj\" (UID: \"a5476f4d-df53-40ad-95d1-c737e8f209fa\") " pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.373928 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.373683    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5476f4d-df53-40ad-95d1-c737e8f209fa-openshift-service-ca\") pod \"maas-api-776cd7fd75-gtnpj\" (UID: \"a5476f4d-df53-40ad-95d1-c737e8f209fa\") " pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.374397 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.374379    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5476f4d-df53-40ad-95d1-c737e8f209fa-openshift-service-ca\") pod \"maas-api-776cd7fd75-gtnpj\" (UID: \"a5476f4d-df53-40ad-95d1-c737e8f209fa\") " pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.376056 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.376037    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5476f4d-df53-40ad-95d1-c737e8f209fa-maas-api-tls\") pod \"maas-api-776cd7fd75-gtnpj\" (UID: \"a5476f4d-df53-40ad-95d1-c737e8f209fa\") " pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.383040 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.383016    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92bdm\" (UniqueName: \"kubernetes.io/projected/a5476f4d-df53-40ad-95d1-c737e8f209fa-kube-api-access-92bdm\") pod \"maas-api-776cd7fd75-gtnpj\" (UID: \"a5476f4d-df53-40ad-95d1-c737e8f209fa\") " pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.425614 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.425571    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:24.566635 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:24.566612    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"]
Jul 23 12:19:24.569230 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:19:24.569206    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5476f4d_df53_40ad_95d1_c737e8f209fa.slice/crio-3c84ada4634602f9dab7035413fd9162b70fbbaeabb690e693fe4ab33777c60f WatchSource:0}: Error finding container 3c84ada4634602f9dab7035413fd9162b70fbbaeabb690e693fe4ab33777c60f: Status 404 returned error can't find the container with id 3c84ada4634602f9dab7035413fd9162b70fbbaeabb690e693fe4ab33777c60f
Jul 23 12:19:25.323841 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:25.323807    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj" event={"ID":"a5476f4d-df53-40ad-95d1-c737e8f209fa","Type":"ContainerStarted","Data":"3c84ada4634602f9dab7035413fd9162b70fbbaeabb690e693fe4ab33777c60f"}
Jul 23 12:19:27.334618 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:27.334582    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj" event={"ID":"a5476f4d-df53-40ad-95d1-c737e8f209fa","Type":"ContainerStarted","Data":"9081c7237ea777128c127298415034096bd87861b226ebbe877b156283142623"}
Jul 23 12:19:27.335021 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:27.334718    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:19:27.355467 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:27.355419    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj" podStartSLOduration=1.4212230940000001 podStartE2EDuration="3.355404342s" totalImagesPullingTime="1.934181248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:19:27.353702994 +0000 UTC m=+551.009212103" watchObservedRunningTime="2026-07-23 12:19:27.355404342 +0000 UTC m=+551.010913169"
Jul 23 12:19:33.344409 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:19:33.344381    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-776cd7fd75-gtnpj"
Jul 23 12:20:09.104100 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.104059    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"]
Jul 23 12:20:09.108918 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.108893    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.110547 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.110515    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.110693 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.110590    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fd37de-6b45-4eae-ad1a-237c47e70305-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.110693 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.110674    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.110842 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.110703    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.110842 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.110740    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjrmf\" (UniqueName: \"kubernetes.io/projected/20fd37de-6b45-4eae-ad1a-237c47e70305-kube-api-access-kjrmf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.110947 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.110871    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.111639 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.111621    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4n8m9\""
Jul 23 12:20:09.111699 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.111636    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 12:20:09.112787 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.112755    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 23 12:20:09.112875 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.112808    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 12:20:09.118058 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.118034    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"]
Jul 23 12:20:09.211483 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.211441    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.211674 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.211488    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.211674 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.211514    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fd37de-6b45-4eae-ad1a-237c47e70305-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.211674 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.211554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.211956 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.211687    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.211956 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.211728    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjrmf\" (UniqueName: \"kubernetes.io/projected/20fd37de-6b45-4eae-ad1a-237c47e70305-kube-api-access-kjrmf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.211956 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.211895    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.211956 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.211922    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.212129 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.211981    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.213863 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.213834    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fd37de-6b45-4eae-ad1a-237c47e70305-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.214179 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.214135    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fd37de-6b45-4eae-ad1a-237c47e70305-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.221153 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.221131    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjrmf\" (UniqueName: \"kubernetes.io/projected/20fd37de-6b45-4eae-ad1a-237c47e70305-kube-api-access-kjrmf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4hwj9\" (UID: \"20fd37de-6b45-4eae-ad1a-237c47e70305\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.421172 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.421078    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:09.560379 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:09.560352    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"]
Jul 23 12:20:09.563342 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:20:09.563288    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20fd37de_6b45_4eae_ad1a_237c47e70305.slice/crio-12930a65c797b317da07f7fd172cdd080d40dffa1d99add587be0cf2fa3b5d0d WatchSource:0}: Error finding container 12930a65c797b317da07f7fd172cdd080d40dffa1d99add587be0cf2fa3b5d0d: Status 404 returned error can't find the container with id 12930a65c797b317da07f7fd172cdd080d40dffa1d99add587be0cf2fa3b5d0d
Jul 23 12:20:10.528382 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:10.528350    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9" event={"ID":"20fd37de-6b45-4eae-ad1a-237c47e70305","Type":"ContainerStarted","Data":"12930a65c797b317da07f7fd172cdd080d40dffa1d99add587be0cf2fa3b5d0d"}
Jul 23 12:20:11.533883 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:11.533850    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9" event={"ID":"20fd37de-6b45-4eae-ad1a-237c47e70305","Type":"ContainerStarted","Data":"4401e6c5a3039442f11be58d1c4faf405e81047cca9580b25f578743fa220b9d"}
Jul 23 12:20:11.555099 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:11.555052    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9" podStartSLOduration=1.3075115290000001 podStartE2EDuration="2.555034543s" totalImagesPullingTime="1.247523014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:20:11.552857842 +0000 UTC m=+595.208366645" watchObservedRunningTime="2026-07-23 12:20:11.555034543 +0000 UTC m=+595.210543347"
Jul 23 12:20:19.421318 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:19.421273    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:19.423733 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:19.421338    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:19.434023 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:19.434001    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:19.585372 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:19.585346    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4hwj9"
Jul 23 12:20:37.482147 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.482116    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"]
Jul 23 12:20:37.485929 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.485909    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.488693 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.488671    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 23 12:20:37.494687 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.494661    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"]
Jul 23 12:20:37.566213 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.566180    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.566382 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.566258    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.566382 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.566301    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6158f94b-9fa9-493f-b480-bfa562dbff56-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.566382 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.566336    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq7w4\" (UniqueName: \"kubernetes.io/projected/6158f94b-9fa9-493f-b480-bfa562dbff56-kube-api-access-mq7w4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.566539 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.566398    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.566539 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.566452    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.667749 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.667668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.667749 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.667711    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.667749 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.667733    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6158f94b-9fa9-493f-b480-bfa562dbff56-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.668035 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.667788    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq7w4\" (UniqueName: \"kubernetes.io/projected/6158f94b-9fa9-493f-b480-bfa562dbff56-kube-api-access-mq7w4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.668035 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.667857    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.668035 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.667900    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.668198 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.668150    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.668198 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.668158    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.668307 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.668273    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.670179 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.670144    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6158f94b-9fa9-493f-b480-bfa562dbff56-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.670315 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.670296    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6158f94b-9fa9-493f-b480-bfa562dbff56-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.677271 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.677244    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq7w4\" (UniqueName: \"kubernetes.io/projected/6158f94b-9fa9-493f-b480-bfa562dbff56-kube-api-access-mq7w4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx\" (UID: \"6158f94b-9fa9-493f-b480-bfa562dbff56\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.796342 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.796256    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:37.930644 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.930621    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"]
Jul 23 12:20:37.933251 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:20:37.933224    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6158f94b_9fa9_493f_b480_bfa562dbff56.slice/crio-ddc685620695def37bcecc1ed83b125f092dc945ff1d0aa7e4678e24823d6be2 WatchSource:0}: Error finding container ddc685620695def37bcecc1ed83b125f092dc945ff1d0aa7e4678e24823d6be2: Status 404 returned error can't find the container with id ddc685620695def37bcecc1ed83b125f092dc945ff1d0aa7e4678e24823d6be2
Jul 23 12:20:37.934941 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:37.934922    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 12:20:38.664406 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:38.664367    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx" event={"ID":"6158f94b-9fa9-493f-b480-bfa562dbff56","Type":"ContainerStarted","Data":"15d59f5a72291cbc86c94ddf9162bae84e12f57b160136d369bf558382bb9532"}
Jul 23 12:20:38.664820 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:38.664420    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx" event={"ID":"6158f94b-9fa9-493f-b480-bfa562dbff56","Type":"ContainerStarted","Data":"ddc685620695def37bcecc1ed83b125f092dc945ff1d0aa7e4678e24823d6be2"}
Jul 23 12:20:38.684612 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:38.684566    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx" podStartSLOduration=1.4840130999999999 podStartE2EDuration="1.684546947s" totalImagesPullingTime="200.533847ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:20:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:20:38.68303213 +0000 UTC m=+622.338540947" watchObservedRunningTime="2026-07-23 12:20:38.684546947 +0000 UTC m=+622.340055795"
Jul 23 12:20:41.079933 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.079892    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"]
Jul 23 12:20:41.083408 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.083386    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.085970 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.085951    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 23 12:20:41.094020 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.093996    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"]
Jul 23 12:20:41.198251 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.198212    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e478984-3678-4efa-92d2-abfeaaf77734-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.198416 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.198261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.198416 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.198282    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.198416 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.198386    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptfqr\" (UniqueName: \"kubernetes.io/projected/5e478984-3678-4efa-92d2-abfeaaf77734-kube-api-access-ptfqr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.198522 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.198446    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.198522 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.198479    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.299532 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.299503    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.299532 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.299541    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.299820 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.299565    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e478984-3678-4efa-92d2-abfeaaf77734-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.299820 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.299723    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.299820 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.299747    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.299820 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.299820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptfqr\" (UniqueName: \"kubernetes.io/projected/5e478984-3678-4efa-92d2-abfeaaf77734-kube-api-access-ptfqr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.300032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.299919    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.300032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.300023    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.300152 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.300088    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.301985 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.301962    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e478984-3678-4efa-92d2-abfeaaf77734-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.302082 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.302027    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e478984-3678-4efa-92d2-abfeaaf77734-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.309010 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.308938    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptfqr\" (UniqueName: \"kubernetes.io/projected/5e478984-3678-4efa-92d2-abfeaaf77734-kube-api-access-ptfqr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rgb4j\" (UID: \"5e478984-3678-4efa-92d2-abfeaaf77734\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.394212 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.394121    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:41.535372 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.535345    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"]
Jul 23 12:20:41.538244 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:20:41.538194    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e478984_3678_4efa_92d2_abfeaaf77734.slice/crio-2f946c27fd0dfcb4c1d303630f190ba2124ae5d83ad305588862618df4abd40a WatchSource:0}: Error finding container 2f946c27fd0dfcb4c1d303630f190ba2124ae5d83ad305588862618df4abd40a: Status 404 returned error can't find the container with id 2f946c27fd0dfcb4c1d303630f190ba2124ae5d83ad305588862618df4abd40a
Jul 23 12:20:41.680052 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:41.679970    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j" event={"ID":"5e478984-3678-4efa-92d2-abfeaaf77734","Type":"ContainerStarted","Data":"2f946c27fd0dfcb4c1d303630f190ba2124ae5d83ad305588862618df4abd40a"}
Jul 23 12:20:42.685808 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:42.685773    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j" event={"ID":"5e478984-3678-4efa-92d2-abfeaaf77734","Type":"ContainerStarted","Data":"82b04ca0bf68aef5d1010529b96c2521eba98a7ff5462010e69f8a7bcdcceab2"}
Jul 23 12:20:42.706569 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:42.706521    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j" podStartSLOduration=1.50023808 podStartE2EDuration="1.706503929s" totalImagesPullingTime="206.265849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:20:42.705465895 +0000 UTC m=+626.360974697" watchObservedRunningTime="2026-07-23 12:20:42.706503929 +0000 UTC m=+626.362012732"
Jul 23 12:20:47.797222 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:47.797187    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:47.797726 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:47.797255    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:47.809526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:47.809496    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:48.726164 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:48.726137    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx"
Jul 23 12:20:51.395255 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:51.395216    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:51.395686 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:51.395279    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:51.407910 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:51.407883    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:20:51.748320 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:20:51.748252    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rgb4j"
Jul 23 12:22:29.703724 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:29.703688    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d79685df6-kx9mx"]
Jul 23 12:22:29.704242 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:29.703912    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-d79685df6-kx9mx" podUID="3934214a-abb1-45f6-be0f-0e81f98c73a5" containerName="authorino" containerID="cri-o://b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2" gracePeriod=30
Jul 23 12:22:29.963252 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:29.963186    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:22:30.055399 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.055363    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3934214a-abb1-45f6-be0f-0e81f98c73a5-tls-cert\") pod \"3934214a-abb1-45f6-be0f-0e81f98c73a5\" (UID: \"3934214a-abb1-45f6-be0f-0e81f98c73a5\") "
Jul 23 12:22:30.055588 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.055425    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmdcm\" (UniqueName: \"kubernetes.io/projected/3934214a-abb1-45f6-be0f-0e81f98c73a5-kube-api-access-nmdcm\") pod \"3934214a-abb1-45f6-be0f-0e81f98c73a5\" (UID: \"3934214a-abb1-45f6-be0f-0e81f98c73a5\") "
Jul 23 12:22:30.057627 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.057592    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3934214a-abb1-45f6-be0f-0e81f98c73a5-kube-api-access-nmdcm" (OuterVolumeSpecName: "kube-api-access-nmdcm") pod "3934214a-abb1-45f6-be0f-0e81f98c73a5" (UID: "3934214a-abb1-45f6-be0f-0e81f98c73a5"). InnerVolumeSpecName "kube-api-access-nmdcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:22:30.065541 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.065514    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3934214a-abb1-45f6-be0f-0e81f98c73a5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3934214a-abb1-45f6-be0f-0e81f98c73a5" (UID: "3934214a-abb1-45f6-be0f-0e81f98c73a5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:22:30.156982 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.156938    2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3934214a-abb1-45f6-be0f-0e81f98c73a5-tls-cert\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:22:30.156982 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.156975    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmdcm\" (UniqueName: \"kubernetes.io/projected/3934214a-abb1-45f6-be0f-0e81f98c73a5-kube-api-access-nmdcm\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:22:30.171408 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.171374    2565 generic.go:356] "Generic (PLEG): container finished" podID="3934214a-abb1-45f6-be0f-0e81f98c73a5" containerID="b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2" exitCode=0
Jul 23 12:22:30.171560 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.171416    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d79685df6-kx9mx"
Jul 23 12:22:30.171560 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.171439    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d79685df6-kx9mx" event={"ID":"3934214a-abb1-45f6-be0f-0e81f98c73a5","Type":"ContainerDied","Data":"b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2"}
Jul 23 12:22:30.171560 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.171470    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d79685df6-kx9mx" event={"ID":"3934214a-abb1-45f6-be0f-0e81f98c73a5","Type":"ContainerDied","Data":"a19f25f1e51075ea01acb2dfdba236b7c6df2239366a582aa7a38af6869e0cf0"}
Jul 23 12:22:30.171560 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.171486    2565 scope.go:117] "RemoveContainer" containerID="b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2"
Jul 23 12:22:30.182887 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.182867    2565 scope.go:117] "RemoveContainer" containerID="b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2"
Jul 23 12:22:30.183168 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:22:30.183147    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2\": container with ID starting with b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2 not found: ID does not exist" containerID="b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2"
Jul 23 12:22:30.183216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.183177    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2"} err="failed to get container status \"b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2\": rpc error: code = NotFound desc = could not find container \"b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2\": container with ID starting with b57964150ce49ee0038a731bd236de949e4cb4b0f2f8d993c19068d74e1a3dd2 not found: ID does not exist"
Jul 23 12:22:30.193264 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.193241    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d79685df6-kx9mx"]
Jul 23 12:22:30.195642 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:30.195618    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d79685df6-kx9mx"]
Jul 23 12:22:31.049635 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:22:31.049594    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3934214a-abb1-45f6-be0f-0e81f98c73a5" path="/var/lib/kubelet/pods/3934214a-abb1-45f6-be0f-0e81f98c73a5/volumes"
Jul 23 12:23:16.470564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:16.470526    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-645984f55d-wfs9p"]
Jul 23 12:23:16.471174 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:16.470802    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-645984f55d-wfs9p" podUID="d0b675bc-c4c1-49ae-94ac-25149d61d33f" containerName="manager" containerID="cri-o://439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b" gracePeriod=30
Jul 23 12:23:16.718748 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:16.718727    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:23:16.759919 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:16.759830    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0b675bc-c4c1-49ae-94ac-25149d61d33f-webhook-cert\") pod \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\" (UID: \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\") "
Jul 23 12:23:16.759919 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:16.759899    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft6jn\" (UniqueName: \"kubernetes.io/projected/d0b675bc-c4c1-49ae-94ac-25149d61d33f-kube-api-access-ft6jn\") pod \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\" (UID: \"d0b675bc-c4c1-49ae-94ac-25149d61d33f\") "
Jul 23 12:23:16.761904 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:16.761874    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0b675bc-c4c1-49ae-94ac-25149d61d33f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d0b675bc-c4c1-49ae-94ac-25149d61d33f" (UID: "d0b675bc-c4c1-49ae-94ac-25149d61d33f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:23:16.762042 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:16.762005    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b675bc-c4c1-49ae-94ac-25149d61d33f-kube-api-access-ft6jn" (OuterVolumeSpecName: "kube-api-access-ft6jn") pod "d0b675bc-c4c1-49ae-94ac-25149d61d33f" (UID: "d0b675bc-c4c1-49ae-94ac-25149d61d33f"). InnerVolumeSpecName "kube-api-access-ft6jn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:23:16.860467 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:16.860422    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft6jn\" (UniqueName: \"kubernetes.io/projected/d0b675bc-c4c1-49ae-94ac-25149d61d33f-kube-api-access-ft6jn\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:23:16.860467 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:16.860463    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0b675bc-c4c1-49ae-94ac-25149d61d33f-webhook-cert\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:23:17.384921 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:17.384839    2565 generic.go:356] "Generic (PLEG): container finished" podID="d0b675bc-c4c1-49ae-94ac-25149d61d33f" containerID="439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b" exitCode=0
Jul 23 12:23:17.384921 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:17.384882    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-wfs9p"
Jul 23 12:23:17.384921 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:17.384915    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-wfs9p" event={"ID":"d0b675bc-c4c1-49ae-94ac-25149d61d33f","Type":"ContainerDied","Data":"439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b"}
Jul 23 12:23:17.385149 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:17.384957    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-wfs9p" event={"ID":"d0b675bc-c4c1-49ae-94ac-25149d61d33f","Type":"ContainerDied","Data":"ae49401561c663e34d6c76f3bf5861826261041bc0552eacba84aa278dd1a86f"}
Jul 23 12:23:17.385149 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:17.384980    2565 scope.go:117] "RemoveContainer" containerID="439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b"
Jul 23 12:23:17.393117 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:17.393100    2565 scope.go:117] "RemoveContainer" containerID="439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b"
Jul 23 12:23:17.393365 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:23:17.393345    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b\": container with ID starting with 439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b not found: ID does not exist" containerID="439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b"
Jul 23 12:23:17.393411 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:17.393373    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b"} err="failed to get container status \"439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b\": rpc error: code = NotFound desc = could not find container \"439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b\": container with ID starting with 439401b0d4f1bafb188ef054cf4f31f101f46d60e6a5a3208000ec0b35fbde3b not found: ID does not exist"
Jul 23 12:23:17.405950 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:17.405920    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-645984f55d-wfs9p"]
Jul 23 12:23:17.411804 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:17.411781    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-645984f55d-wfs9p"]
Jul 23 12:23:19.049875 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:23:19.049842    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0b675bc-c4c1-49ae-94ac-25149d61d33f" path="/var/lib/kubelet/pods/d0b675bc-c4c1-49ae-94ac-25149d61d33f/volumes"
Jul 23 12:24:18.274695 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.274661    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-645984f55d-8mfl7"]
Jul 23 12:24:18.275195 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.275028    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3934214a-abb1-45f6-be0f-0e81f98c73a5" containerName="authorino"
Jul 23 12:24:18.275195 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.275039    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3934214a-abb1-45f6-be0f-0e81f98c73a5" containerName="authorino"
Jul 23 12:24:18.275195 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.275058    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0b675bc-c4c1-49ae-94ac-25149d61d33f" containerName="manager"
Jul 23 12:24:18.275195 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.275064    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0b675bc-c4c1-49ae-94ac-25149d61d33f" containerName="manager"
Jul 23 12:24:18.275195 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.275119    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0b675bc-c4c1-49ae-94ac-25149d61d33f" containerName="manager"
Jul 23 12:24:18.275195 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.275128    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3934214a-abb1-45f6-be0f-0e81f98c73a5" containerName="authorino"
Jul 23 12:24:18.277954 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.277938    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:18.282241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.282218    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 12:24:18.282426 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.282270    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 12:24:18.282426 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.282349    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9rcs9\""
Jul 23 12:24:18.287538 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.287517    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645984f55d-8mfl7"]
Jul 23 12:24:18.393062 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.393017    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4k2n\" (UniqueName: \"kubernetes.io/projected/f3864ae2-ddad-4463-b03e-c433737a3c3e-kube-api-access-h4k2n\") pod \"maas-controller-645984f55d-8mfl7\" (UID: \"f3864ae2-ddad-4463-b03e-c433737a3c3e\") " pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:18.393237 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.393087    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3864ae2-ddad-4463-b03e-c433737a3c3e-webhook-cert\") pod \"maas-controller-645984f55d-8mfl7\" (UID: \"f3864ae2-ddad-4463-b03e-c433737a3c3e\") " pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:18.494455 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.494416    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4k2n\" (UniqueName: \"kubernetes.io/projected/f3864ae2-ddad-4463-b03e-c433737a3c3e-kube-api-access-h4k2n\") pod \"maas-controller-645984f55d-8mfl7\" (UID: \"f3864ae2-ddad-4463-b03e-c433737a3c3e\") " pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:18.494652 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.494470    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3864ae2-ddad-4463-b03e-c433737a3c3e-webhook-cert\") pod \"maas-controller-645984f55d-8mfl7\" (UID: \"f3864ae2-ddad-4463-b03e-c433737a3c3e\") " pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:18.496960 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.496936    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3864ae2-ddad-4463-b03e-c433737a3c3e-webhook-cert\") pod \"maas-controller-645984f55d-8mfl7\" (UID: \"f3864ae2-ddad-4463-b03e-c433737a3c3e\") " pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:18.505144 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.505121    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4k2n\" (UniqueName: \"kubernetes.io/projected/f3864ae2-ddad-4463-b03e-c433737a3c3e-kube-api-access-h4k2n\") pod \"maas-controller-645984f55d-8mfl7\" (UID: \"f3864ae2-ddad-4463-b03e-c433737a3c3e\") " pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:18.588272 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.588239    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:18.727314 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:18.727290    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645984f55d-8mfl7"]
Jul 23 12:24:18.729822 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:24:18.729790    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3864ae2_ddad_4463_b03e_c433737a3c3e.slice/crio-6c631d5210831447b3bfe821a95dcfd704fb887f0baf25213c713fc160ea4130 WatchSource:0}: Error finding container 6c631d5210831447b3bfe821a95dcfd704fb887f0baf25213c713fc160ea4130: Status 404 returned error can't find the container with id 6c631d5210831447b3bfe821a95dcfd704fb887f0baf25213c713fc160ea4130
Jul 23 12:24:19.662508 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:19.662421    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-8mfl7" event={"ID":"f3864ae2-ddad-4463-b03e-c433737a3c3e","Type":"ContainerStarted","Data":"c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3"}
Jul 23 12:24:19.662508 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:19.662467    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-8mfl7" event={"ID":"f3864ae2-ddad-4463-b03e-c433737a3c3e","Type":"ContainerStarted","Data":"6c631d5210831447b3bfe821a95dcfd704fb887f0baf25213c713fc160ea4130"}
Jul 23 12:24:19.662508 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:19.662497    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:19.682064 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:19.682017    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-645984f55d-8mfl7" podStartSLOduration=1.052150582 podStartE2EDuration="1.681999237s" totalImagesPullingTime="629.848655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:24:19.680510161 +0000 UTC m=+843.336018964" watchObservedRunningTime="2026-07-23 12:24:19.681999237 +0000 UTC m=+843.337508040"
Jul 23 12:24:30.668749 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:30.668711    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:35.354672 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.354632    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-645984f55d-8mfl7"]
Jul 23 12:24:35.355208 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.354872    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-645984f55d-8mfl7" podUID="f3864ae2-ddad-4463-b03e-c433737a3c3e" containerName="manager" containerID="cri-o://c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3" gracePeriod=30
Jul 23 12:24:35.607894 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.607837    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:35.735147 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.735111    2565 generic.go:356] "Generic (PLEG): container finished" podID="f3864ae2-ddad-4463-b03e-c433737a3c3e" containerID="c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3" exitCode=0
Jul 23 12:24:35.735147 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.735151    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-8mfl7"
Jul 23 12:24:35.735363 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.735196    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-8mfl7" event={"ID":"f3864ae2-ddad-4463-b03e-c433737a3c3e","Type":"ContainerDied","Data":"c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3"}
Jul 23 12:24:35.735363 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.735278    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-8mfl7" event={"ID":"f3864ae2-ddad-4463-b03e-c433737a3c3e","Type":"ContainerDied","Data":"6c631d5210831447b3bfe821a95dcfd704fb887f0baf25213c713fc160ea4130"}
Jul 23 12:24:35.735363 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.735300    2565 scope.go:117] "RemoveContainer" containerID="c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3"
Jul 23 12:24:35.743562 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.743539    2565 scope.go:117] "RemoveContainer" containerID="c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3"
Jul 23 12:24:35.743878 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:24:35.743848    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3\": container with ID starting with c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3 not found: ID does not exist" containerID="c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3"
Jul 23 12:24:35.743963 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.743892    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3"} err="failed to get container status \"c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3\": rpc error: code = NotFound desc = could not find container \"c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3\": container with ID starting with c943e5ff7fd596eba4cc0376e9f03d5f7a9bbcbd178d824a896aa0690832c7b3 not found: ID does not exist"
Jul 23 12:24:35.746134 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.746116    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4k2n\" (UniqueName: \"kubernetes.io/projected/f3864ae2-ddad-4463-b03e-c433737a3c3e-kube-api-access-h4k2n\") pod \"f3864ae2-ddad-4463-b03e-c433737a3c3e\" (UID: \"f3864ae2-ddad-4463-b03e-c433737a3c3e\") "
Jul 23 12:24:35.746206 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.746150    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3864ae2-ddad-4463-b03e-c433737a3c3e-webhook-cert\") pod \"f3864ae2-ddad-4463-b03e-c433737a3c3e\" (UID: \"f3864ae2-ddad-4463-b03e-c433737a3c3e\") "
Jul 23 12:24:35.748107 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.748084    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3864ae2-ddad-4463-b03e-c433737a3c3e-kube-api-access-h4k2n" (OuterVolumeSpecName: "kube-api-access-h4k2n") pod "f3864ae2-ddad-4463-b03e-c433737a3c3e" (UID: "f3864ae2-ddad-4463-b03e-c433737a3c3e"). InnerVolumeSpecName "kube-api-access-h4k2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:24:35.748203 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.748176    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3864ae2-ddad-4463-b03e-c433737a3c3e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f3864ae2-ddad-4463-b03e-c433737a3c3e" (UID: "f3864ae2-ddad-4463-b03e-c433737a3c3e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:24:35.847697 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.847662    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4k2n\" (UniqueName: \"kubernetes.io/projected/f3864ae2-ddad-4463-b03e-c433737a3c3e-kube-api-access-h4k2n\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:24:35.847697 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:35.847693    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3864ae2-ddad-4463-b03e-c433737a3c3e-webhook-cert\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:24:36.059772 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.059735    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-645984f55d-8mfl7"]
Jul 23 12:24:36.062075 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.062051    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-645984f55d-8mfl7"]
Jul 23 12:24:36.890627 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.890591    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-645984f55d-sdxx7"]
Jul 23 12:24:36.891019 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.890952    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3864ae2-ddad-4463-b03e-c433737a3c3e" containerName="manager"
Jul 23 12:24:36.891019 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.890964    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3864ae2-ddad-4463-b03e-c433737a3c3e" containerName="manager"
Jul 23 12:24:36.891086 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.891022    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3864ae2-ddad-4463-b03e-c433737a3c3e" containerName="manager"
Jul 23 12:24:36.895307 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.895287    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:24:36.899286 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.899268    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9rcs9\""
Jul 23 12:24:36.899399 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.899267    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 12:24:36.899399 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.899270    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 12:24:36.904267 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:36.904248    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645984f55d-sdxx7"]
Jul 23 12:24:37.050276 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.050232    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3864ae2-ddad-4463-b03e-c433737a3c3e" path="/var/lib/kubelet/pods/f3864ae2-ddad-4463-b03e-c433737a3c3e/volumes"
Jul 23 12:24:37.058140 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.058113    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsfdh\" (UniqueName: \"kubernetes.io/projected/cf388995-0283-40ec-9ce5-5b989989e4ef-kube-api-access-dsfdh\") pod \"maas-controller-645984f55d-sdxx7\" (UID: \"cf388995-0283-40ec-9ce5-5b989989e4ef\") " pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:24:37.058285 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.058194    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf388995-0283-40ec-9ce5-5b989989e4ef-webhook-cert\") pod \"maas-controller-645984f55d-sdxx7\" (UID: \"cf388995-0283-40ec-9ce5-5b989989e4ef\") " pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:24:37.158649 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.158566    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsfdh\" (UniqueName: \"kubernetes.io/projected/cf388995-0283-40ec-9ce5-5b989989e4ef-kube-api-access-dsfdh\") pod \"maas-controller-645984f55d-sdxx7\" (UID: \"cf388995-0283-40ec-9ce5-5b989989e4ef\") " pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:24:37.158833 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.158677    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf388995-0283-40ec-9ce5-5b989989e4ef-webhook-cert\") pod \"maas-controller-645984f55d-sdxx7\" (UID: \"cf388995-0283-40ec-9ce5-5b989989e4ef\") " pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:24:37.161002 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.160979    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf388995-0283-40ec-9ce5-5b989989e4ef-webhook-cert\") pod \"maas-controller-645984f55d-sdxx7\" (UID: \"cf388995-0283-40ec-9ce5-5b989989e4ef\") " pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:24:37.168639 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.168616    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsfdh\" (UniqueName: \"kubernetes.io/projected/cf388995-0283-40ec-9ce5-5b989989e4ef-kube-api-access-dsfdh\") pod \"maas-controller-645984f55d-sdxx7\" (UID: \"cf388995-0283-40ec-9ce5-5b989989e4ef\") " pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:24:37.206783 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.206730    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:24:37.342023 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.341991    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645984f55d-sdxx7"]
Jul 23 12:24:37.343238 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:24:37.343213    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf388995_0283_40ec_9ce5_5b989989e4ef.slice/crio-d4feaf73f25e87b7052c120049ff72c8e360bb6d46db3094bdbda59f6128583a WatchSource:0}: Error finding container d4feaf73f25e87b7052c120049ff72c8e360bb6d46db3094bdbda59f6128583a: Status 404 returned error can't find the container with id d4feaf73f25e87b7052c120049ff72c8e360bb6d46db3094bdbda59f6128583a
Jul 23 12:24:37.747413 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:37.747389    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-sdxx7" event={"ID":"cf388995-0283-40ec-9ce5-5b989989e4ef","Type":"ContainerStarted","Data":"d4feaf73f25e87b7052c120049ff72c8e360bb6d46db3094bdbda59f6128583a"}
Jul 23 12:24:38.753224 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:38.753191    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645984f55d-sdxx7" event={"ID":"cf388995-0283-40ec-9ce5-5b989989e4ef","Type":"ContainerStarted","Data":"b4fed3658d8781c1ce040f4c79cb456b75a089929b7b558f5e6ca1a09c6e894f"}
Jul 23 12:24:38.753613 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:38.753308    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:24:38.773729 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:38.773685    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-645984f55d-sdxx7" podStartSLOduration=2.377262934 podStartE2EDuration="2.773669349s" totalImagesPullingTime="396.406415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:24:38.772609151 +0000 UTC m=+862.428117957" watchObservedRunningTime="2026-07-23 12:24:38.773669349 +0000 UTC m=+862.429178153"
Jul 23 12:24:49.759891 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:24:49.759860    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-645984f55d-sdxx7"
Jul 23 12:29:20.597905 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:20.597865    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"]
Jul 23 12:29:20.598496 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:20.598135    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4" podUID="917678ef-368e-4107-8534-66330a604db2" containerName="manager" containerID="cri-o://313cabf393e8eb9239a18c647952ff26ce0b9c9b860db5df1d9a1393de5700ae" gracePeriod=10
Jul 23 12:29:23.791435 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:23.791384    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4" podUID="917678ef-368e-4107-8534-66330a604db2" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.29:8081/readyz\": dial tcp 10.134.0.29:8081: connect: connection refused"
Jul 23 12:29:24.038350 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:24.038316    2565 generic.go:356] "Generic (PLEG): container finished" podID="917678ef-368e-4107-8534-66330a604db2" containerID="313cabf393e8eb9239a18c647952ff26ce0b9c9b860db5df1d9a1393de5700ae" exitCode=0
Jul 23 12:29:24.038526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:24.038391    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4" event={"ID":"917678ef-368e-4107-8534-66330a604db2","Type":"ContainerDied","Data":"313cabf393e8eb9239a18c647952ff26ce0b9c9b860db5df1d9a1393de5700ae"}
Jul 23 12:29:24.451660 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:24.451637    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:29:24.620994 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:24.620903    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917678ef-368e-4107-8534-66330a604db2-extensions-socket-volume\") pod \"917678ef-368e-4107-8534-66330a604db2\" (UID: \"917678ef-368e-4107-8534-66330a604db2\") "
Jul 23 12:29:24.620994 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:24.620961    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5f7z7\" (UniqueName: \"kubernetes.io/projected/917678ef-368e-4107-8534-66330a604db2-kube-api-access-5f7z7\") pod \"917678ef-368e-4107-8534-66330a604db2\" (UID: \"917678ef-368e-4107-8534-66330a604db2\") "
Jul 23 12:29:24.621282 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:24.621257    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/917678ef-368e-4107-8534-66330a604db2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "917678ef-368e-4107-8534-66330a604db2" (UID: "917678ef-368e-4107-8534-66330a604db2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:29:24.623123 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:24.623101    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/917678ef-368e-4107-8534-66330a604db2-kube-api-access-5f7z7" (OuterVolumeSpecName: "kube-api-access-5f7z7") pod "917678ef-368e-4107-8534-66330a604db2" (UID: "917678ef-368e-4107-8534-66330a604db2"). InnerVolumeSpecName "kube-api-access-5f7z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:29:24.721495 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:24.721456    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917678ef-368e-4107-8534-66330a604db2-extensions-socket-volume\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:29:24.721495 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:24.721484    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5f7z7\" (UniqueName: \"kubernetes.io/projected/917678ef-368e-4107-8534-66330a604db2-kube-api-access-5f7z7\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:29:25.045053 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:25.044975    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"
Jul 23 12:29:25.050829 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:25.050799    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4" event={"ID":"917678ef-368e-4107-8534-66330a604db2","Type":"ContainerDied","Data":"1d98bd628118264e7f1c3f0b79aad4091cfac573b14f3e819ed050ec7d998f9b"}
Jul 23 12:29:25.050980 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:25.050856    2565 scope.go:117] "RemoveContainer" containerID="313cabf393e8eb9239a18c647952ff26ce0b9c9b860db5df1d9a1393de5700ae"
Jul 23 12:29:25.071693 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:25.071664    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"]
Jul 23 12:29:25.080506 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:25.080486    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xw8v4"]
Jul 23 12:29:27.049329 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:29:27.049253    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="917678ef-368e-4107-8534-66330a604db2" path="/var/lib/kubelet/pods/917678ef-368e-4107-8534-66330a604db2/volumes"
Jul 23 12:30:00.153824 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.153721    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6"]
Jul 23 12:30:00.154284 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.154150    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="917678ef-368e-4107-8534-66330a604db2" containerName="manager"
Jul 23 12:30:00.154284 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.154164    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="917678ef-368e-4107-8534-66330a604db2" containerName="manager"
Jul 23 12:30:00.154284 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.154220    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="917678ef-368e-4107-8534-66330a604db2" containerName="manager"
Jul 23 12:30:00.157148 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.157132    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6"
Jul 23 12:30:00.160209 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.160181    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:30:00.165733 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.165691    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6"]
Jul 23 12:30:00.167932 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.167914    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6"
Jul 23 12:30:00.305571 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.305550    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6"]
Jul 23 12:30:00.308083 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:30:00.308054    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fde351_87dd_4982_9c42_b35ad86c2512.slice/crio-64a8eb9152676fcdb90eb63fc3e2d880ae7c6f440679cbbf3a7ff98a8a9d5307 WatchSource:0}: Error finding container 64a8eb9152676fcdb90eb63fc3e2d880ae7c6f440679cbbf3a7ff98a8a9d5307: Status 404 returned error can't find the container with id 64a8eb9152676fcdb90eb63fc3e2d880ae7c6f440679cbbf3a7ff98a8a9d5307
Jul 23 12:30:00.309999 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:00.309985    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 12:30:01.203813 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:01.203770    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6" event={"ID":"03fde351-87dd-4982-9c42-b35ad86c2512","Type":"ContainerStarted","Data":"64a8eb9152676fcdb90eb63fc3e2d880ae7c6f440679cbbf3a7ff98a8a9d5307"}
Jul 23 12:30:03.220228 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:03.220198    2565 generic.go:356] "Generic (PLEG): container finished" podID="03fde351-87dd-4982-9c42-b35ad86c2512" containerID="18e12f648816bb52beed24d32dd6f44b7b17e8c84bc580e1899c9ec90c50538e" exitCode=0
Jul 23 12:30:03.220603 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:03.220284    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6" event={"ID":"03fde351-87dd-4982-9c42-b35ad86c2512","Type":"ContainerDied","Data":"18e12f648816bb52beed24d32dd6f44b7b17e8c84bc580e1899c9ec90c50538e"}
Jul 23 12:30:04.349183 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:04.349158    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6"
Jul 23 12:30:05.230836 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:05.230800    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6"
Jul 23 12:30:05.230836 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:05.230825    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746830-xn5g6" event={"ID":"03fde351-87dd-4982-9c42-b35ad86c2512","Type":"ContainerDied","Data":"64a8eb9152676fcdb90eb63fc3e2d880ae7c6f440679cbbf3a7ff98a8a9d5307"}
Jul 23 12:30:05.231041 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:05.230879    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64a8eb9152676fcdb90eb63fc3e2d880ae7c6f440679cbbf3a7ff98a8a9d5307"
Jul 23 12:30:21.800941 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:21.800892    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"]
Jul 23 12:30:21.801323 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:21.801239    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03fde351-87dd-4982-9c42-b35ad86c2512" containerName="cleanup"
Jul 23 12:30:21.801323 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:21.801250    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="03fde351-87dd-4982-9c42-b35ad86c2512" containerName="cleanup"
Jul 23 12:30:21.801323 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:21.801301    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="03fde351-87dd-4982-9c42-b35ad86c2512" containerName="cleanup"
Jul 23 12:30:21.804357 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:21.804338    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:30:21.807790 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:21.807771    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5tb7f\""
Jul 23 12:30:21.817750 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:21.817727    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"]
Jul 23 12:30:21.900710 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:21.900671    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vf4ht\" (UID: \"1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:30:21.900923 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:21.900749    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xq5l\" (UniqueName: \"kubernetes.io/projected/1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e-kube-api-access-4xq5l\") pod \"kuadrant-operator-controller-manager-86878c769f-vf4ht\" (UID: \"1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:30:22.002131 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:22.002099    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xq5l\" (UniqueName: \"kubernetes.io/projected/1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e-kube-api-access-4xq5l\") pod \"kuadrant-operator-controller-manager-86878c769f-vf4ht\" (UID: \"1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:30:22.002296 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:22.002175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vf4ht\" (UID: \"1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:30:22.002573 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:22.002550    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vf4ht\" (UID: \"1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:30:22.013010 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:22.012982    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xq5l\" (UniqueName: \"kubernetes.io/projected/1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e-kube-api-access-4xq5l\") pod \"kuadrant-operator-controller-manager-86878c769f-vf4ht\" (UID: \"1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:30:22.114063 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:22.114008    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:30:22.255508 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:22.255465    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"]
Jul 23 12:30:22.257826 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:30:22.257794    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f45b2a5_05b1_4c92_9aa2_40a2aaa9840e.slice/crio-373acf9f016095103198d79c0e0125c8be39ec9c7b7664b8144cfa4063419d6b WatchSource:0}: Error finding container 373acf9f016095103198d79c0e0125c8be39ec9c7b7664b8144cfa4063419d6b: Status 404 returned error can't find the container with id 373acf9f016095103198d79c0e0125c8be39ec9c7b7664b8144cfa4063419d6b
Jul 23 12:30:22.305799 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:22.305754    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht" event={"ID":"1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e","Type":"ContainerStarted","Data":"373acf9f016095103198d79c0e0125c8be39ec9c7b7664b8144cfa4063419d6b"}
Jul 23 12:30:23.314855 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:23.314815    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht" event={"ID":"1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e","Type":"ContainerStarted","Data":"1b7c70bc04ab5e87a7fea4733187b3ae2534ad49dd9b3162b868eca47950589d"}
Jul 23 12:30:23.315246 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:23.314948    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:30:23.336710 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:23.336663    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht" podStartSLOduration=2.336646144 podStartE2EDuration="2.336646144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:30:23.335295443 +0000 UTC m=+1206.990804246" watchObservedRunningTime="2026-07-23 12:30:23.336646144 +0000 UTC m=+1206.992154949"
Jul 23 12:30:34.321340 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:30:34.321313    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vf4ht"
Jul 23 12:37:43.043678 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.043646    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"]
Jul 23 12:37:43.047212 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.047185    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.050581 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.050561    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-543ade38-serving-cert\""
Jul 23 12:37:43.060195 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.060163    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"]
Jul 23 12:37:43.186228 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.186190    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a3654a-7500-40f7-9fd5-65fb7cb091b5-maas-api-tls\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.186228 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.186228    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smxhk\" (UniqueName: \"kubernetes.io/projected/30a3654a-7500-40f7-9fd5-65fb7cb091b5-kube-api-access-smxhk\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.186460 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.186253    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a3654a-7500-40f7-9fd5-65fb7cb091b5-openshift-service-ca\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.287226 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.287185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a3654a-7500-40f7-9fd5-65fb7cb091b5-maas-api-tls\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.287226 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.287223    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smxhk\" (UniqueName: \"kubernetes.io/projected/30a3654a-7500-40f7-9fd5-65fb7cb091b5-kube-api-access-smxhk\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.287457 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.287246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a3654a-7500-40f7-9fd5-65fb7cb091b5-openshift-service-ca\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.287457 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:37:43.287340    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-serving-cert: secret "maas-api-e2e-ait-543ade38-serving-cert" not found
Jul 23 12:37:43.287457 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:37:43.287401    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30a3654a-7500-40f7-9fd5-65fb7cb091b5-maas-api-tls podName:30a3654a-7500-40f7-9fd5-65fb7cb091b5 nodeName:}" failed. No retries permitted until 2026-07-23 12:37:43.787385963 +0000 UTC m=+1647.442894748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/30a3654a-7500-40f7-9fd5-65fb7cb091b5-maas-api-tls") pod "maas-api-e2e-ait-543ade38-978c695b9-vlnpr" (UID: "30a3654a-7500-40f7-9fd5-65fb7cb091b5") : secret "maas-api-e2e-ait-543ade38-serving-cert" not found
Jul 23 12:37:43.288021 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.288003    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a3654a-7500-40f7-9fd5-65fb7cb091b5-openshift-service-ca\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.301123 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.301047    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smxhk\" (UniqueName: \"kubernetes.io/projected/30a3654a-7500-40f7-9fd5-65fb7cb091b5-kube-api-access-smxhk\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.790957 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.790917    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a3654a-7500-40f7-9fd5-65fb7cb091b5-maas-api-tls\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.793361 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.793338    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a3654a-7500-40f7-9fd5-65fb7cb091b5-maas-api-tls\") pod \"maas-api-e2e-ait-543ade38-978c695b9-vlnpr\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:43.957692 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:43.957661    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:44.106342 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:44.106317    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"]
Jul 23 12:37:44.109116 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:37:44.109086    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a3654a_7500_40f7_9fd5_65fb7cb091b5.slice/crio-039bf61868d32dfae521bc614dcb376e31b75cb7925207e354b8191d687ded4f WatchSource:0}: Error finding container 039bf61868d32dfae521bc614dcb376e31b75cb7925207e354b8191d687ded4f: Status 404 returned error can't find the container with id 039bf61868d32dfae521bc614dcb376e31b75cb7925207e354b8191d687ded4f
Jul 23 12:37:44.110595 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:44.110576    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 12:37:44.334655 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:44.334620    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr" event={"ID":"30a3654a-7500-40f7-9fd5-65fb7cb091b5","Type":"ContainerStarted","Data":"039bf61868d32dfae521bc614dcb376e31b75cb7925207e354b8191d687ded4f"}
Jul 23 12:37:46.345450 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:46.345413    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr" event={"ID":"30a3654a-7500-40f7-9fd5-65fb7cb091b5","Type":"ContainerStarted","Data":"360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060"}
Jul 23 12:37:46.345895 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:46.345519    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:37:46.365099 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:46.365050    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr" podStartSLOduration=1.8635754260000001 podStartE2EDuration="3.365033015s" totalImagesPullingTime="1.501457589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:37:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:37:46.363372873 +0000 UTC m=+1650.018881677" watchObservedRunningTime="2026-07-23 12:37:46.365033015 +0000 UTC m=+1650.020541818"
Jul 23 12:37:52.357320 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:37:52.357287    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:38:18.923159 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:18.923119    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"]
Jul 23 12:38:18.923670 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:18.923369    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr" podUID="30a3654a-7500-40f7-9fd5-65fb7cb091b5" containerName="maas-api" containerID="cri-o://360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060" gracePeriod=30
Jul 23 12:38:19.176407 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.176332    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:38:19.302103 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.302053    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smxhk\" (UniqueName: \"kubernetes.io/projected/30a3654a-7500-40f7-9fd5-65fb7cb091b5-kube-api-access-smxhk\") pod \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") "
Jul 23 12:38:19.302288 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.302145    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a3654a-7500-40f7-9fd5-65fb7cb091b5-maas-api-tls\") pod \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") "
Jul 23 12:38:19.302288 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.302169    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a3654a-7500-40f7-9fd5-65fb7cb091b5-openshift-service-ca\") pod \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\" (UID: \"30a3654a-7500-40f7-9fd5-65fb7cb091b5\") "
Jul 23 12:38:19.302604 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.302571    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a3654a-7500-40f7-9fd5-65fb7cb091b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30a3654a-7500-40f7-9fd5-65fb7cb091b5" (UID: "30a3654a-7500-40f7-9fd5-65fb7cb091b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:38:19.304397 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.304362    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30a3654a-7500-40f7-9fd5-65fb7cb091b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30a3654a-7500-40f7-9fd5-65fb7cb091b5" (UID: "30a3654a-7500-40f7-9fd5-65fb7cb091b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:38:19.304397 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.304388    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a3654a-7500-40f7-9fd5-65fb7cb091b5-kube-api-access-smxhk" (OuterVolumeSpecName: "kube-api-access-smxhk") pod "30a3654a-7500-40f7-9fd5-65fb7cb091b5" (UID: "30a3654a-7500-40f7-9fd5-65fb7cb091b5"). InnerVolumeSpecName "kube-api-access-smxhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:38:19.403034 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.402991    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a3654a-7500-40f7-9fd5-65fb7cb091b5-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:38:19.403034 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.403028    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a3654a-7500-40f7-9fd5-65fb7cb091b5-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:38:19.403034 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.403041    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smxhk\" (UniqueName: \"kubernetes.io/projected/30a3654a-7500-40f7-9fd5-65fb7cb091b5-kube-api-access-smxhk\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:38:19.498913 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.498815    2565 generic.go:356] "Generic (PLEG): container finished" podID="30a3654a-7500-40f7-9fd5-65fb7cb091b5" containerID="360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060" exitCode=0
Jul 23 12:38:19.498913 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.498862    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"
Jul 23 12:38:19.499155 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.498908    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr" event={"ID":"30a3654a-7500-40f7-9fd5-65fb7cb091b5","Type":"ContainerDied","Data":"360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060"}
Jul 23 12:38:19.499155 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.498967    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr" event={"ID":"30a3654a-7500-40f7-9fd5-65fb7cb091b5","Type":"ContainerDied","Data":"039bf61868d32dfae521bc614dcb376e31b75cb7925207e354b8191d687ded4f"}
Jul 23 12:38:19.499155 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.498997    2565 scope.go:117] "RemoveContainer" containerID="360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060"
Jul 23 12:38:19.508161 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.508138    2565 scope.go:117] "RemoveContainer" containerID="360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060"
Jul 23 12:38:19.508497 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:38:19.508477    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060\": container with ID starting with 360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060 not found: ID does not exist" containerID="360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060"
Jul 23 12:38:19.508583 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.508505    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060"} err="failed to get container status \"360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060\": rpc error: code = NotFound desc = could not find container \"360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060\": container with ID starting with 360cb72ca38b52afa6e8caae57f8972c1a4031db5d98a99216df4f579cb3a060 not found: ID does not exist"
Jul 23 12:38:19.522466 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.522427    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"]
Jul 23 12:38:19.527028 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:19.526997    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-543ade38-978c695b9-vlnpr"]
Jul 23 12:38:21.050197 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:21.050164    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30a3654a-7500-40f7-9fd5-65fb7cb091b5" path="/var/lib/kubelet/pods/30a3654a-7500-40f7-9fd5-65fb7cb091b5/volumes"
Jul 23 12:38:27.143356 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.143320    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"]
Jul 23 12:38:27.143779 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.143699    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a3654a-7500-40f7-9fd5-65fb7cb091b5" containerName="maas-api"
Jul 23 12:38:27.143779 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.143713    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a3654a-7500-40f7-9fd5-65fb7cb091b5" containerName="maas-api"
Jul 23 12:38:27.143862 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.143794    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a3654a-7500-40f7-9fd5-65fb7cb091b5" containerName="maas-api"
Jul 23 12:38:27.148474 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.148452    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.151158 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.151139    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-2cd5664e-serving-cert\""
Jul 23 12:38:27.161699 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.161661    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"]
Jul 23 12:38:27.271977 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.271932    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b972ef-952e-499d-9842-bf0343a7a060-maas-api-tls\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.271977 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.271977    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b972ef-952e-499d-9842-bf0343a7a060-openshift-service-ca\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.272203 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.272063    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2dfh\" (UniqueName: \"kubernetes.io/projected/81b972ef-952e-499d-9842-bf0343a7a060-kube-api-access-g2dfh\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.373087 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.373052    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2dfh\" (UniqueName: \"kubernetes.io/projected/81b972ef-952e-499d-9842-bf0343a7a060-kube-api-access-g2dfh\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.373267 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.373140    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b972ef-952e-499d-9842-bf0343a7a060-maas-api-tls\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.373267 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.373169    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b972ef-952e-499d-9842-bf0343a7a060-openshift-service-ca\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.373347 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:38:27.373273    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-serving-cert: secret "maas-api-e2e-migrate-2cd5664e-serving-cert" not found
Jul 23 12:38:27.373347 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:38:27.373344    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b972ef-952e-499d-9842-bf0343a7a060-maas-api-tls podName:81b972ef-952e-499d-9842-bf0343a7a060 nodeName:}" failed. No retries permitted until 2026-07-23 12:38:27.873328655 +0000 UTC m=+1691.528837441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81b972ef-952e-499d-9842-bf0343a7a060-maas-api-tls") pod "maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5" (UID: "81b972ef-952e-499d-9842-bf0343a7a060") : secret "maas-api-e2e-migrate-2cd5664e-serving-cert" not found
Jul 23 12:38:27.373810 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.373795    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b972ef-952e-499d-9842-bf0343a7a060-openshift-service-ca\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.384830 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.384803    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2dfh\" (UniqueName: \"kubernetes.io/projected/81b972ef-952e-499d-9842-bf0343a7a060-kube-api-access-g2dfh\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.878111 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.878074    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b972ef-952e-499d-9842-bf0343a7a060-maas-api-tls\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:27.880593 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:27.880561    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b972ef-952e-499d-9842-bf0343a7a060-maas-api-tls\") pod \"maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:28.068280 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:28.068232    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:28.209599 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:28.209574    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"]
Jul 23 12:38:28.212251 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:38:28.212224    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b972ef_952e_499d_9842_bf0343a7a060.slice/crio-ce7b91e76b114012d1241e97b5ec2c086fc1445f123120163f97b3591b62300a WatchSource:0}: Error finding container ce7b91e76b114012d1241e97b5ec2c086fc1445f123120163f97b3591b62300a: Status 404 returned error can't find the container with id ce7b91e76b114012d1241e97b5ec2c086fc1445f123120163f97b3591b62300a
Jul 23 12:38:28.545500 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:28.545408    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5" event={"ID":"81b972ef-952e-499d-9842-bf0343a7a060","Type":"ContainerStarted","Data":"ce7b91e76b114012d1241e97b5ec2c086fc1445f123120163f97b3591b62300a"}
Jul 23 12:38:29.550699 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:29.550662    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5" event={"ID":"81b972ef-952e-499d-9842-bf0343a7a060","Type":"ContainerStarted","Data":"c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64"}
Jul 23 12:38:29.551133 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:29.550731    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:29.570427 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:29.570367    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5" podStartSLOduration=2.190949207 podStartE2EDuration="2.570349255s" totalImagesPullingTime="379.400048ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:38:29.569245322 +0000 UTC m=+1693.224754126" watchObservedRunningTime="2026-07-23 12:38:29.570349255 +0000 UTC m=+1693.225858059"
Jul 23 12:38:35.561701 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:35.561670    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:38:52.017612 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.017524    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"]
Jul 23 12:38:52.021352 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.021334    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"
Jul 23 12:38:52.023940 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.023920    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:38:52.027966 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.027925    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"]
Jul 23 12:38:52.085696 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.085664    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4778aa51-7360-4e78-a2ca-ee7e716df8b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs\" (UID: \"4778aa51-7360-4e78-a2ca-ee7e716df8b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"
Jul 23 12:38:52.186161 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.186110    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4778aa51-7360-4e78-a2ca-ee7e716df8b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs\" (UID: \"4778aa51-7360-4e78-a2ca-ee7e716df8b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"
Jul 23 12:38:52.186789 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.186750    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4778aa51-7360-4e78-a2ca-ee7e716df8b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs\" (UID: \"4778aa51-7360-4e78-a2ca-ee7e716df8b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"
Jul 23 12:38:52.333438 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.333401    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"
Jul 23 12:38:52.474960 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.474936    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"]
Jul 23 12:38:52.478562 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:38:52.478530    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4778aa51_7360_4e78_a2ca_ee7e716df8b7.slice/crio-7575159d09ebbc12c32909b72b10c8b7a7e5c87c4bd49dc412fb529f0ba7e7b5 WatchSource:0}: Error finding container 7575159d09ebbc12c32909b72b10c8b7a7e5c87c4bd49dc412fb529f0ba7e7b5: Status 404 returned error can't find the container with id 7575159d09ebbc12c32909b72b10c8b7a7e5c87c4bd49dc412fb529f0ba7e7b5
Jul 23 12:38:52.662194 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:52.662140    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs" event={"ID":"4778aa51-7360-4e78-a2ca-ee7e716df8b7","Type":"ContainerStarted","Data":"7575159d09ebbc12c32909b72b10c8b7a7e5c87c4bd49dc412fb529f0ba7e7b5"}
Jul 23 12:38:53.667646 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:53.667608    2565 generic.go:356] "Generic (PLEG): container finished" podID="4778aa51-7360-4e78-a2ca-ee7e716df8b7" containerID="4df7f9122612d50cabf49285263e2392497ffecfc447fc801d42c1e016fa52a7" exitCode=0
Jul 23 12:38:53.668040 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:53.667693    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs" event={"ID":"4778aa51-7360-4e78-a2ca-ee7e716df8b7","Type":"ContainerDied","Data":"4df7f9122612d50cabf49285263e2392497ffecfc447fc801d42c1e016fa52a7"}
Jul 23 12:38:54.809179 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:54.809154    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"
Jul 23 12:38:54.910396 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:54.910356    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4778aa51-7360-4e78-a2ca-ee7e716df8b7-maas-api-service-ca\") pod \"4778aa51-7360-4e78-a2ca-ee7e716df8b7\" (UID: \"4778aa51-7360-4e78-a2ca-ee7e716df8b7\") "
Jul 23 12:38:54.910726 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:54.910701    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4778aa51-7360-4e78-a2ca-ee7e716df8b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4778aa51-7360-4e78-a2ca-ee7e716df8b7" (UID: "4778aa51-7360-4e78-a2ca-ee7e716df8b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:38:55.011636 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:55.011546    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4778aa51-7360-4e78-a2ca-ee7e716df8b7-maas-api-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:38:55.678817 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:55.678779    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs" event={"ID":"4778aa51-7360-4e78-a2ca-ee7e716df8b7","Type":"ContainerDied","Data":"7575159d09ebbc12c32909b72b10c8b7a7e5c87c4bd49dc412fb529f0ba7e7b5"}
Jul 23 12:38:55.678817 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:55.678804    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"
Jul 23 12:38:55.679023 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:38:55.678839    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7575159d09ebbc12c32909b72b10c8b7a7e5c87c4bd49dc412fb529f0ba7e7b5"
Jul 23 12:39:02.744639 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:02.744606    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"]
Jul 23 12:39:02.745199 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:02.744837    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5" podUID="81b972ef-952e-499d-9842-bf0343a7a060" containerName="maas-api" containerID="cri-o://c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64" gracePeriod=30
Jul 23 12:39:02.998446 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:02.998380    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:39:03.081102 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.081069    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2dfh\" (UniqueName: \"kubernetes.io/projected/81b972ef-952e-499d-9842-bf0343a7a060-kube-api-access-g2dfh\") pod \"81b972ef-952e-499d-9842-bf0343a7a060\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") "
Jul 23 12:39:03.081308 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.081164    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b972ef-952e-499d-9842-bf0343a7a060-openshift-service-ca\") pod \"81b972ef-952e-499d-9842-bf0343a7a060\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") "
Jul 23 12:39:03.081308 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.081193    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b972ef-952e-499d-9842-bf0343a7a060-maas-api-tls\") pod \"81b972ef-952e-499d-9842-bf0343a7a060\" (UID: \"81b972ef-952e-499d-9842-bf0343a7a060\") "
Jul 23 12:39:03.081557 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.081528    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b972ef-952e-499d-9842-bf0343a7a060-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81b972ef-952e-499d-9842-bf0343a7a060" (UID: "81b972ef-952e-499d-9842-bf0343a7a060"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:39:03.083283 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.083254    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b972ef-952e-499d-9842-bf0343a7a060-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81b972ef-952e-499d-9842-bf0343a7a060" (UID: "81b972ef-952e-499d-9842-bf0343a7a060"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:39:03.083427 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.083407    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b972ef-952e-499d-9842-bf0343a7a060-kube-api-access-g2dfh" (OuterVolumeSpecName: "kube-api-access-g2dfh") pod "81b972ef-952e-499d-9842-bf0343a7a060" (UID: "81b972ef-952e-499d-9842-bf0343a7a060"). InnerVolumeSpecName "kube-api-access-g2dfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:39:03.182771 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.182733    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2dfh\" (UniqueName: \"kubernetes.io/projected/81b972ef-952e-499d-9842-bf0343a7a060-kube-api-access-g2dfh\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:39:03.182953 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.182790    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b972ef-952e-499d-9842-bf0343a7a060-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:39:03.182953 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.182801    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b972ef-952e-499d-9842-bf0343a7a060-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:39:03.715717 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.715682    2565 generic.go:356] "Generic (PLEG): container finished" podID="81b972ef-952e-499d-9842-bf0343a7a060" containerID="c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64" exitCode=0
Jul 23 12:39:03.715964 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.715728    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"
Jul 23 12:39:03.715964 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.715787    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5" event={"ID":"81b972ef-952e-499d-9842-bf0343a7a060","Type":"ContainerDied","Data":"c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64"}
Jul 23 12:39:03.715964 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.715842    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5" event={"ID":"81b972ef-952e-499d-9842-bf0343a7a060","Type":"ContainerDied","Data":"ce7b91e76b114012d1241e97b5ec2c086fc1445f123120163f97b3591b62300a"}
Jul 23 12:39:03.715964 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.715865    2565 scope.go:117] "RemoveContainer" containerID="c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64"
Jul 23 12:39:03.724832 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.724814    2565 scope.go:117] "RemoveContainer" containerID="c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64"
Jul 23 12:39:03.725109 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:39:03.725090    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64\": container with ID starting with c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64 not found: ID does not exist" containerID="c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64"
Jul 23 12:39:03.725180 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.725118    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64"} err="failed to get container status \"c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64\": rpc error: code = NotFound desc = could not find container \"c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64\": container with ID starting with c533365eca780a3ead0ebbb9c26b3ecf75a11255da0cf4b035ea00dd7726bb64 not found: ID does not exist"
Jul 23 12:39:03.739499 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.739463    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"]
Jul 23 12:39:03.743364 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:03.743332    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2cd5664e-6bfcc98856-kk7k5"]
Jul 23 12:39:05.049774 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:05.049724    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81b972ef-952e-499d-9842-bf0343a7a060" path="/var/lib/kubelet/pods/81b972ef-952e-499d-9842-bf0343a7a060/volumes"
Jul 23 12:39:07.107621 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:07.107590    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"]
Jul 23 12:39:07.110995 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:07.110969    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2cd5664e-2mwvs"]
Jul 23 12:39:09.050585 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:09.050551    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4778aa51-7360-4e78-a2ca-ee7e716df8b7" path="/var/lib/kubelet/pods/4778aa51-7360-4e78-a2ca-ee7e716df8b7/volumes"
Jul 23 12:39:10.370280 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.370243    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"]
Jul 23 12:39:10.370742 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.370603    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4778aa51-7360-4e78-a2ca-ee7e716df8b7" containerName="revoke-keys"
Jul 23 12:39:10.370742 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.370614    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4778aa51-7360-4e78-a2ca-ee7e716df8b7" containerName="revoke-keys"
Jul 23 12:39:10.370742 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.370631    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b972ef-952e-499d-9842-bf0343a7a060" containerName="maas-api"
Jul 23 12:39:10.370742 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.370636    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b972ef-952e-499d-9842-bf0343a7a060" containerName="maas-api"
Jul 23 12:39:10.370742 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.370688    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4778aa51-7360-4e78-a2ca-ee7e716df8b7" containerName="revoke-keys"
Jul 23 12:39:10.370742 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.370699    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b972ef-952e-499d-9842-bf0343a7a060" containerName="maas-api"
Jul 23 12:39:10.375163 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.375142    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:10.377677 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.377652    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9d7c1e5c-serving-cert\""
Jul 23 12:39:10.385977 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.385940    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"]
Jul 23 12:39:10.449487 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.449456    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bdbd5f6-720e-41c5-a399-a49f732781d8-maas-api-tls\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:10.449487 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.449491    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2srlz\" (UniqueName: \"kubernetes.io/projected/1bdbd5f6-720e-41c5-a399-a49f732781d8-kube-api-access-2srlz\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:10.449685 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.449627    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bdbd5f6-720e-41c5-a399-a49f732781d8-openshift-service-ca\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:10.551129 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.551093    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bdbd5f6-720e-41c5-a399-a49f732781d8-openshift-service-ca\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:10.551299 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.551164    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bdbd5f6-720e-41c5-a399-a49f732781d8-maas-api-tls\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:10.551299 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.551192    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2srlz\" (UniqueName: \"kubernetes.io/projected/1bdbd5f6-720e-41c5-a399-a49f732781d8-kube-api-access-2srlz\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:10.551371 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:39:10.551314    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-serving-cert: secret "maas-api-e2e-ait-9d7c1e5c-serving-cert" not found
Jul 23 12:39:10.551405 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:39:10.551377    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bdbd5f6-720e-41c5-a399-a49f732781d8-maas-api-tls podName:1bdbd5f6-720e-41c5-a399-a49f732781d8 nodeName:}" failed. No retries permitted until 2026-07-23 12:39:11.051362299 +0000 UTC m=+1734.706871081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1bdbd5f6-720e-41c5-a399-a49f732781d8-maas-api-tls") pod "maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj" (UID: "1bdbd5f6-720e-41c5-a399-a49f732781d8") : secret "maas-api-e2e-ait-9d7c1e5c-serving-cert" not found
Jul 23 12:39:10.551986 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.551970    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bdbd5f6-720e-41c5-a399-a49f732781d8-openshift-service-ca\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:10.562998 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:10.562970    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2srlz\" (UniqueName: \"kubernetes.io/projected/1bdbd5f6-720e-41c5-a399-a49f732781d8-kube-api-access-2srlz\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:11.055775 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:11.055731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bdbd5f6-720e-41c5-a399-a49f732781d8-maas-api-tls\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:11.058302 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:11.058274    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bdbd5f6-720e-41c5-a399-a49f732781d8-maas-api-tls\") pod \"maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:11.286161 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:11.286111    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:11.432120 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:11.432067    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"]
Jul 23 12:39:11.434608 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:39:11.434574    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bdbd5f6_720e_41c5_a399_a49f732781d8.slice/crio-d4313bed6847ba6cff2d283242360193de9d0260c1c6614c79433ee1ff9ae2db WatchSource:0}: Error finding container d4313bed6847ba6cff2d283242360193de9d0260c1c6614c79433ee1ff9ae2db: Status 404 returned error can't find the container with id d4313bed6847ba6cff2d283242360193de9d0260c1c6614c79433ee1ff9ae2db
Jul 23 12:39:11.756033 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:11.755949    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj" event={"ID":"1bdbd5f6-720e-41c5-a399-a49f732781d8","Type":"ContainerStarted","Data":"d4313bed6847ba6cff2d283242360193de9d0260c1c6614c79433ee1ff9ae2db"}
Jul 23 12:39:12.761989 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:12.761948    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj" event={"ID":"1bdbd5f6-720e-41c5-a399-a49f732781d8","Type":"ContainerStarted","Data":"caf2f2cffd13151df26436d5c0c8ee8b803d8444a870ca0386eae853c8e02f12"}
Jul 23 12:39:12.762385 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:12.762044    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:12.782411 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:12.782355    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj" podStartSLOduration=2.317440777 podStartE2EDuration="2.782339228s" totalImagesPullingTime="464.898451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:39:12.781960378 +0000 UTC m=+1736.437469182" watchObservedRunningTime="2026-07-23 12:39:12.782339228 +0000 UTC m=+1736.437848032"
Jul 23 12:39:18.772912 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:18.772883    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:36.196174 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.196111    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"]
Jul 23 12:39:36.200812 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.200794    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"
Jul 23 12:39:36.204231 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.204182    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:39:36.207069 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.207041    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"]
Jul 23 12:39:36.275966 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.275935    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/354c8b9b-dbbb-416b-a1de-bdc5e1b44c93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp\" (UID: \"354c8b9b-dbbb-416b-a1de-bdc5e1b44c93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"
Jul 23 12:39:36.377398 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.377361    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/354c8b9b-dbbb-416b-a1de-bdc5e1b44c93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp\" (UID: \"354c8b9b-dbbb-416b-a1de-bdc5e1b44c93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"
Jul 23 12:39:36.378115 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.378095    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/354c8b9b-dbbb-416b-a1de-bdc5e1b44c93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp\" (UID: \"354c8b9b-dbbb-416b-a1de-bdc5e1b44c93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"
Jul 23 12:39:36.513868 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.513747    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"
Jul 23 12:39:36.654376 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.654349    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"]
Jul 23 12:39:36.657513 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:39:36.657483    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod354c8b9b_dbbb_416b_a1de_bdc5e1b44c93.slice/crio-779e9102fde215e2e5adf01208cfc6aac20a7ef867a44e18cf0275842d44b28c WatchSource:0}: Error finding container 779e9102fde215e2e5adf01208cfc6aac20a7ef867a44e18cf0275842d44b28c: Status 404 returned error can't find the container with id 779e9102fde215e2e5adf01208cfc6aac20a7ef867a44e18cf0275842d44b28c
Jul 23 12:39:36.881624 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:36.881586    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp" event={"ID":"354c8b9b-dbbb-416b-a1de-bdc5e1b44c93","Type":"ContainerStarted","Data":"779e9102fde215e2e5adf01208cfc6aac20a7ef867a44e18cf0275842d44b28c"}
Jul 23 12:39:37.887356 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:37.887316    2565 generic.go:356] "Generic (PLEG): container finished" podID="354c8b9b-dbbb-416b-a1de-bdc5e1b44c93" containerID="264ed42c75662a7c760ecf2c3f8240809403376f6de826299b1d9824a445f082" exitCode=0
Jul 23 12:39:37.887732 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:37.887398    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp" event={"ID":"354c8b9b-dbbb-416b-a1de-bdc5e1b44c93","Type":"ContainerDied","Data":"264ed42c75662a7c760ecf2c3f8240809403376f6de826299b1d9824a445f082"}
Jul 23 12:39:39.028288 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:39.028263    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"
Jul 23 12:39:39.102936 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:39.102895    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/354c8b9b-dbbb-416b-a1de-bdc5e1b44c93-maas-api-service-ca\") pod \"354c8b9b-dbbb-416b-a1de-bdc5e1b44c93\" (UID: \"354c8b9b-dbbb-416b-a1de-bdc5e1b44c93\") "
Jul 23 12:39:39.103252 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:39.103230    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/354c8b9b-dbbb-416b-a1de-bdc5e1b44c93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "354c8b9b-dbbb-416b-a1de-bdc5e1b44c93" (UID: "354c8b9b-dbbb-416b-a1de-bdc5e1b44c93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:39:39.204049 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:39.203963    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/354c8b9b-dbbb-416b-a1de-bdc5e1b44c93-maas-api-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:39:39.898622 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:39.898596    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"
Jul 23 12:39:39.898810 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:39.898595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp" event={"ID":"354c8b9b-dbbb-416b-a1de-bdc5e1b44c93","Type":"ContainerDied","Data":"779e9102fde215e2e5adf01208cfc6aac20a7ef867a44e18cf0275842d44b28c"}
Jul 23 12:39:39.898810 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:39.898711    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="779e9102fde215e2e5adf01208cfc6aac20a7ef867a44e18cf0275842d44b28c"
Jul 23 12:39:46.691313 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:46.691279    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"]
Jul 23 12:39:46.691793 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:46.691519    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj" podUID="1bdbd5f6-720e-41c5-a399-a49f732781d8" containerName="maas-api" containerID="cri-o://caf2f2cffd13151df26436d5c0c8ee8b803d8444a870ca0386eae853c8e02f12" gracePeriod=30
Jul 23 12:39:46.935813 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:46.935782    2565 generic.go:356] "Generic (PLEG): container finished" podID="1bdbd5f6-720e-41c5-a399-a49f732781d8" containerID="caf2f2cffd13151df26436d5c0c8ee8b803d8444a870ca0386eae853c8e02f12" exitCode=0
Jul 23 12:39:46.935942 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:46.935850    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj" event={"ID":"1bdbd5f6-720e-41c5-a399-a49f732781d8","Type":"ContainerDied","Data":"caf2f2cffd13151df26436d5c0c8ee8b803d8444a870ca0386eae853c8e02f12"}
Jul 23 12:39:46.935942 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:46.935913    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj" event={"ID":"1bdbd5f6-720e-41c5-a399-a49f732781d8","Type":"ContainerDied","Data":"d4313bed6847ba6cff2d283242360193de9d0260c1c6614c79433ee1ff9ae2db"}
Jul 23 12:39:46.935942 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:46.935937    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4313bed6847ba6cff2d283242360193de9d0260c1c6614c79433ee1ff9ae2db"
Jul 23 12:39:46.947878 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:46.947819    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:47.074534 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.074496    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bdbd5f6-720e-41c5-a399-a49f732781d8-maas-api-tls\") pod \"1bdbd5f6-720e-41c5-a399-a49f732781d8\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") "
Jul 23 12:39:47.074699 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.074573    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2srlz\" (UniqueName: \"kubernetes.io/projected/1bdbd5f6-720e-41c5-a399-a49f732781d8-kube-api-access-2srlz\") pod \"1bdbd5f6-720e-41c5-a399-a49f732781d8\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") "
Jul 23 12:39:47.074699 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.074591    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bdbd5f6-720e-41c5-a399-a49f732781d8-openshift-service-ca\") pod \"1bdbd5f6-720e-41c5-a399-a49f732781d8\" (UID: \"1bdbd5f6-720e-41c5-a399-a49f732781d8\") "
Jul 23 12:39:47.075022 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.074991    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bdbd5f6-720e-41c5-a399-a49f732781d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1bdbd5f6-720e-41c5-a399-a49f732781d8" (UID: "1bdbd5f6-720e-41c5-a399-a49f732781d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:39:47.076602 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.076582    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bdbd5f6-720e-41c5-a399-a49f732781d8-kube-api-access-2srlz" (OuterVolumeSpecName: "kube-api-access-2srlz") pod "1bdbd5f6-720e-41c5-a399-a49f732781d8" (UID: "1bdbd5f6-720e-41c5-a399-a49f732781d8"). InnerVolumeSpecName "kube-api-access-2srlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:39:47.076667 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.076634    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bdbd5f6-720e-41c5-a399-a49f732781d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1bdbd5f6-720e-41c5-a399-a49f732781d8" (UID: "1bdbd5f6-720e-41c5-a399-a49f732781d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:39:47.175508 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.175471    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2srlz\" (UniqueName: \"kubernetes.io/projected/1bdbd5f6-720e-41c5-a399-a49f732781d8-kube-api-access-2srlz\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:39:47.175508 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.175500    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bdbd5f6-720e-41c5-a399-a49f732781d8-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:39:47.175508 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.175510    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bdbd5f6-720e-41c5-a399-a49f732781d8-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:39:47.940547 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.940517    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"
Jul 23 12:39:47.965240 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.965201    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"]
Jul 23 12:39:47.970816 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:47.970787    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9d7c1e5c-7b6f59f557-rdhxj"]
Jul 23 12:39:49.051521 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:49.051481    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bdbd5f6-720e-41c5-a399-a49f732781d8" path="/var/lib/kubelet/pods/1bdbd5f6-720e-41c5-a399-a49f732781d8/volumes"
Jul 23 12:39:51.310230 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:51.310196    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"]
Jul 23 12:39:51.315822 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:51.315791    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d7c1e5c-hp9fp"]
Jul 23 12:39:53.050667 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:53.050632    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="354c8b9b-dbbb-416b-a1de-bdc5e1b44c93" path="/var/lib/kubelet/pods/354c8b9b-dbbb-416b-a1de-bdc5e1b44c93/volumes"
Jul 23 12:39:55.487394 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.487356    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"]
Jul 23 12:39:55.487803 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.487708    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bdbd5f6-720e-41c5-a399-a49f732781d8" containerName="maas-api"
Jul 23 12:39:55.487803 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.487719    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bdbd5f6-720e-41c5-a399-a49f732781d8" containerName="maas-api"
Jul 23 12:39:55.487803 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.487735    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354c8b9b-dbbb-416b-a1de-bdc5e1b44c93" containerName="revoke-keys"
Jul 23 12:39:55.487803 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.487741    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="354c8b9b-dbbb-416b-a1de-bdc5e1b44c93" containerName="revoke-keys"
Jul 23 12:39:55.487995 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.487813    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bdbd5f6-720e-41c5-a399-a49f732781d8" containerName="maas-api"
Jul 23 12:39:55.487995 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.487822    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="354c8b9b-dbbb-416b-a1de-bdc5e1b44c93" containerName="revoke-keys"
Jul 23 12:39:55.490990 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.490970    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.493913 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.493889    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-3e72e548-serving-cert\""
Jul 23 12:39:55.501303 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.501269    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"]
Jul 23 12:39:55.552146 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.552114    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92443773-5313-4ab1-a368-7a0554eb7a96-maas-api-tls\") pod \"maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.552310 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.552168    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92443773-5313-4ab1-a368-7a0554eb7a96-openshift-service-ca\") pod \"maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.552310 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.552197    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2nww\" (UniqueName: \"kubernetes.io/projected/92443773-5313-4ab1-a368-7a0554eb7a96-kube-api-access-j2nww\") pod \"maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.653486 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.653444    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92443773-5313-4ab1-a368-7a0554eb7a96-maas-api-tls\") pod \"maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.653690 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.653497    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92443773-5313-4ab1-a368-7a0554eb7a96-openshift-service-ca\") pod \"maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.653690 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.653522    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2nww\" (UniqueName: \"kubernetes.io/projected/92443773-5313-4ab1-a368-7a0554eb7a96-kube-api-access-j2nww\") pod \"maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.654253 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.654233    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92443773-5313-4ab1-a368-7a0554eb7a96-openshift-service-ca\") pod \"maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.656046 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.656011    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92443773-5313-4ab1-a368-7a0554eb7a96-maas-api-tls\") pod \"maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.663653 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.663630    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2nww\" (UniqueName: \"kubernetes.io/projected/92443773-5313-4ab1-a368-7a0554eb7a96-kube-api-access-j2nww\") pod \"maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.801261 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.801167    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:55.940179 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.940150    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"]
Jul 23 12:39:55.942587 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:39:55.942557    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92443773_5313_4ab1_a368_7a0554eb7a96.slice/crio-a84cb03da5431349e176314681cd3c3dcc3d3a8d1a5048413ee6f83b8a98188a WatchSource:0}: Error finding container a84cb03da5431349e176314681cd3c3dcc3d3a8d1a5048413ee6f83b8a98188a: Status 404 returned error can't find the container with id a84cb03da5431349e176314681cd3c3dcc3d3a8d1a5048413ee6f83b8a98188a
Jul 23 12:39:55.991247 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:55.991206    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw" event={"ID":"92443773-5313-4ab1-a368-7a0554eb7a96","Type":"ContainerStarted","Data":"a84cb03da5431349e176314681cd3c3dcc3d3a8d1a5048413ee6f83b8a98188a"}
Jul 23 12:39:56.997941 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:56.997910    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw" event={"ID":"92443773-5313-4ab1-a368-7a0554eb7a96","Type":"ContainerStarted","Data":"099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc"}
Jul 23 12:39:56.998351 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:56.998032    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:39:57.024083 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:39:57.024026    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw" podStartSLOduration=1.576526917 podStartE2EDuration="2.023987572s" totalImagesPullingTime="447.460655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:39:57.023010216 +0000 UTC m=+1780.678519021" watchObservedRunningTime="2026-07-23 12:39:57.023987572 +0000 UTC m=+1780.679496375"
Jul 23 12:40:03.008098 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:03.008064    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:40:30.829007 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:30.828973    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"]
Jul 23 12:40:30.829493 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:30.829187    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw" podUID="92443773-5313-4ab1-a368-7a0554eb7a96" containerName="maas-api" containerID="cri-o://099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc" gracePeriod=30
Jul 23 12:40:31.083277 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.083213    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:40:31.157074 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.157040    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2nww\" (UniqueName: \"kubernetes.io/projected/92443773-5313-4ab1-a368-7a0554eb7a96-kube-api-access-j2nww\") pod \"92443773-5313-4ab1-a368-7a0554eb7a96\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") "
Jul 23 12:40:31.157074 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.157081    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92443773-5313-4ab1-a368-7a0554eb7a96-openshift-service-ca\") pod \"92443773-5313-4ab1-a368-7a0554eb7a96\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") "
Jul 23 12:40:31.157335 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.157111    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92443773-5313-4ab1-a368-7a0554eb7a96-maas-api-tls\") pod \"92443773-5313-4ab1-a368-7a0554eb7a96\" (UID: \"92443773-5313-4ab1-a368-7a0554eb7a96\") "
Jul 23 12:40:31.157444 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.157420    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92443773-5313-4ab1-a368-7a0554eb7a96-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92443773-5313-4ab1-a368-7a0554eb7a96" (UID: "92443773-5313-4ab1-a368-7a0554eb7a96"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:40:31.159231 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.159204    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92443773-5313-4ab1-a368-7a0554eb7a96-kube-api-access-j2nww" (OuterVolumeSpecName: "kube-api-access-j2nww") pod "92443773-5313-4ab1-a368-7a0554eb7a96" (UID: "92443773-5313-4ab1-a368-7a0554eb7a96"). InnerVolumeSpecName "kube-api-access-j2nww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:40:31.159329 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.159206    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92443773-5313-4ab1-a368-7a0554eb7a96-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92443773-5313-4ab1-a368-7a0554eb7a96" (UID: "92443773-5313-4ab1-a368-7a0554eb7a96"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:40:31.169810 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.169783    2565 generic.go:356] "Generic (PLEG): container finished" podID="92443773-5313-4ab1-a368-7a0554eb7a96" containerID="099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc" exitCode=0
Jul 23 12:40:31.169936 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.169828    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw" event={"ID":"92443773-5313-4ab1-a368-7a0554eb7a96","Type":"ContainerDied","Data":"099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc"}
Jul 23 12:40:31.169936 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.169880    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"
Jul 23 12:40:31.169936 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.169899    2565 scope.go:117] "RemoveContainer" containerID="099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc"
Jul 23 12:40:31.170084 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.169885    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw" event={"ID":"92443773-5313-4ab1-a368-7a0554eb7a96","Type":"ContainerDied","Data":"a84cb03da5431349e176314681cd3c3dcc3d3a8d1a5048413ee6f83b8a98188a"}
Jul 23 12:40:31.179003 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.178986    2565 scope.go:117] "RemoveContainer" containerID="099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc"
Jul 23 12:40:31.179276 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:40:31.179257    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc\": container with ID starting with 099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc not found: ID does not exist" containerID="099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc"
Jul 23 12:40:31.179325 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.179286    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc"} err="failed to get container status \"099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc\": rpc error: code = NotFound desc = could not find container \"099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc\": container with ID starting with 099d55a0d01659c687e8e917108e08c809b8d5baab262bc76773a03debb493fc not found: ID does not exist"
Jul 23 12:40:31.193137 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.193109    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"]
Jul 23 12:40:31.196185 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.196163    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3e72e548-7d4c79d67d-xd9bw"]
Jul 23 12:40:31.257870 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.257836    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2nww\" (UniqueName: \"kubernetes.io/projected/92443773-5313-4ab1-a368-7a0554eb7a96-kube-api-access-j2nww\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:40:31.257870 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.257862    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92443773-5313-4ab1-a368-7a0554eb7a96-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:40:31.257870 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:31.257873    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92443773-5313-4ab1-a368-7a0554eb7a96-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:40:33.050362 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:40:33.050317    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92443773-5313-4ab1-a368-7a0554eb7a96" path="/var/lib/kubelet/pods/92443773-5313-4ab1-a368-7a0554eb7a96/volumes"
Jul 23 12:42:26.154685 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.154648    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"]
Jul 23 12:42:26.155255 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.155043    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92443773-5313-4ab1-a368-7a0554eb7a96" containerName="maas-api"
Jul 23 12:42:26.155255 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.155059    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="92443773-5313-4ab1-a368-7a0554eb7a96" containerName="maas-api"
Jul 23 12:42:26.155255 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.155165    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="92443773-5313-4ab1-a368-7a0554eb7a96" containerName="maas-api"
Jul 23 12:42:26.157906 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.157875    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.160664 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.160641    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-312a3fbf-serving-cert\""
Jul 23 12:42:26.170429 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.170397    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"]
Jul 23 12:42:26.249636 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.249596    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416aaeab-a8db-432d-ae00-2756d05b8507-openshift-service-ca\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.249636 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.249640    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416aaeab-a8db-432d-ae00-2756d05b8507-maas-api-tls\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.249915 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.249711    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpvw6\" (UniqueName: \"kubernetes.io/projected/416aaeab-a8db-432d-ae00-2756d05b8507-kube-api-access-zpvw6\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.351015 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.350972    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416aaeab-a8db-432d-ae00-2756d05b8507-openshift-service-ca\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.351015 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.351019    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416aaeab-a8db-432d-ae00-2756d05b8507-maas-api-tls\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.351247 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.351052    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpvw6\" (UniqueName: \"kubernetes.io/projected/416aaeab-a8db-432d-ae00-2756d05b8507-kube-api-access-zpvw6\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.351247 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:42:26.351189    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-serving-cert: secret "maas-api-e2e-mt-312a3fbf-serving-cert" not found
Jul 23 12:42:26.351322 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:42:26.351269    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416aaeab-a8db-432d-ae00-2756d05b8507-maas-api-tls podName:416aaeab-a8db-432d-ae00-2756d05b8507 nodeName:}" failed. No retries permitted until 2026-07-23 12:42:26.851249924 +0000 UTC m=+1930.506758706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/416aaeab-a8db-432d-ae00-2756d05b8507-maas-api-tls") pod "maas-api-e2e-mt-312a3fbf-55cc886467-29qh5" (UID: "416aaeab-a8db-432d-ae00-2756d05b8507") : secret "maas-api-e2e-mt-312a3fbf-serving-cert" not found
Jul 23 12:42:26.351584 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.351566    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416aaeab-a8db-432d-ae00-2756d05b8507-openshift-service-ca\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.362428 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.362403    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpvw6\" (UniqueName: \"kubernetes.io/projected/416aaeab-a8db-432d-ae00-2756d05b8507-kube-api-access-zpvw6\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.856055 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.856016    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416aaeab-a8db-432d-ae00-2756d05b8507-maas-api-tls\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:26.858550 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:26.858523    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416aaeab-a8db-432d-ae00-2756d05b8507-maas-api-tls\") pod \"maas-api-e2e-mt-312a3fbf-55cc886467-29qh5\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:27.069357 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:27.069322    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:27.219685 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:27.219643    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"]
Jul 23 12:42:27.222671 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:42:27.222643    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416aaeab_a8db_432d_ae00_2756d05b8507.slice/crio-f91e0aa6175d54ffd8ee277c1edee162b85007b5aef51e2e1c3e0f490fe4af9d WatchSource:0}: Error finding container f91e0aa6175d54ffd8ee277c1edee162b85007b5aef51e2e1c3e0f490fe4af9d: Status 404 returned error can't find the container with id f91e0aa6175d54ffd8ee277c1edee162b85007b5aef51e2e1c3e0f490fe4af9d
Jul 23 12:42:27.712077 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:27.712041    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5" event={"ID":"416aaeab-a8db-432d-ae00-2756d05b8507","Type":"ContainerStarted","Data":"f91e0aa6175d54ffd8ee277c1edee162b85007b5aef51e2e1c3e0f490fe4af9d"}
Jul 23 12:42:28.719003 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:28.718967    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5" event={"ID":"416aaeab-a8db-432d-ae00-2756d05b8507","Type":"ContainerStarted","Data":"144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d"}
Jul 23 12:42:28.719417 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:28.719034    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:28.743940 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:28.743875    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5" podStartSLOduration=2.276543136 podStartE2EDuration="2.743857316s" totalImagesPullingTime="467.31418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:42:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:42:28.741901771 +0000 UTC m=+1932.397410574" watchObservedRunningTime="2026-07-23 12:42:28.743857316 +0000 UTC m=+1932.399366119"
Jul 23 12:42:34.730036 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:34.730008    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:38.567217 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:38.567185    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"]
Jul 23 12:42:38.570717 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:38.570698    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"
Jul 23 12:42:38.573669 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:38.573647    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:42:38.579341 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:38.579315    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"]
Jul 23 12:42:38.657567 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:38.657523    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8f0d283-fdb2-4bcc-9f15-746f99aaf231-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf\" (UID: \"c8f0d283-fdb2-4bcc-9f15-746f99aaf231\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"
Jul 23 12:42:38.758301 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:38.758260    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8f0d283-fdb2-4bcc-9f15-746f99aaf231-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf\" (UID: \"c8f0d283-fdb2-4bcc-9f15-746f99aaf231\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"
Jul 23 12:42:38.758913 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:38.758894    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8f0d283-fdb2-4bcc-9f15-746f99aaf231-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf\" (UID: \"c8f0d283-fdb2-4bcc-9f15-746f99aaf231\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"
Jul 23 12:42:38.882314 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:38.882225    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"
Jul 23 12:42:39.024398 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:39.024376    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"]
Jul 23 12:42:39.026589 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:42:39.026565    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8f0d283_fdb2_4bcc_9f15_746f99aaf231.slice/crio-6612a5ffa4b0f49df18b686cd405c2bf32571c91ff0943d93a1b80b89d6840c9 WatchSource:0}: Error finding container 6612a5ffa4b0f49df18b686cd405c2bf32571c91ff0943d93a1b80b89d6840c9: Status 404 returned error can't find the container with id 6612a5ffa4b0f49df18b686cd405c2bf32571c91ff0943d93a1b80b89d6840c9
Jul 23 12:42:39.773052 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:39.773014    2565 generic.go:356] "Generic (PLEG): container finished" podID="c8f0d283-fdb2-4bcc-9f15-746f99aaf231" containerID="c8881aa5b2948163c6eab39e37b0a6623654c45ce2dbf4c2fcc5afbe7d21adeb" exitCode=0
Jul 23 12:42:39.773576 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:39.773098    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf" event={"ID":"c8f0d283-fdb2-4bcc-9f15-746f99aaf231","Type":"ContainerDied","Data":"c8881aa5b2948163c6eab39e37b0a6623654c45ce2dbf4c2fcc5afbe7d21adeb"}
Jul 23 12:42:39.773576 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:39.773155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf" event={"ID":"c8f0d283-fdb2-4bcc-9f15-746f99aaf231","Type":"ContainerStarted","Data":"6612a5ffa4b0f49df18b686cd405c2bf32571c91ff0943d93a1b80b89d6840c9"}
Jul 23 12:42:40.909250 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:40.909224    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"
Jul 23 12:42:40.977880 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:40.977844    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8f0d283-fdb2-4bcc-9f15-746f99aaf231-maas-api-service-ca\") pod \"c8f0d283-fdb2-4bcc-9f15-746f99aaf231\" (UID: \"c8f0d283-fdb2-4bcc-9f15-746f99aaf231\") "
Jul 23 12:42:40.978200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:40.978177    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8f0d283-fdb2-4bcc-9f15-746f99aaf231-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8f0d283-fdb2-4bcc-9f15-746f99aaf231" (UID: "c8f0d283-fdb2-4bcc-9f15-746f99aaf231"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:42:41.079022 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:41.078984    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8f0d283-fdb2-4bcc-9f15-746f99aaf231-maas-api-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:42:41.783143 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:41.783105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf" event={"ID":"c8f0d283-fdb2-4bcc-9f15-746f99aaf231","Type":"ContainerDied","Data":"6612a5ffa4b0f49df18b686cd405c2bf32571c91ff0943d93a1b80b89d6840c9"}
Jul 23 12:42:41.783143 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:41.783134    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"
Jul 23 12:42:41.783359 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:41.783165    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6612a5ffa4b0f49df18b686cd405c2bf32571c91ff0943d93a1b80b89d6840c9"
Jul 23 12:42:48.901857 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:48.901822    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"]
Jul 23 12:42:48.902261 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:48.902017    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5" podUID="416aaeab-a8db-432d-ae00-2756d05b8507" containerName="maas-api" containerID="cri-o://144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d" gracePeriod=30
Jul 23 12:42:49.157618 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.157557    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:49.250696 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.250648    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416aaeab-a8db-432d-ae00-2756d05b8507-openshift-service-ca\") pod \"416aaeab-a8db-432d-ae00-2756d05b8507\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") "
Jul 23 12:42:49.250696 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.250706    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpvw6\" (UniqueName: \"kubernetes.io/projected/416aaeab-a8db-432d-ae00-2756d05b8507-kube-api-access-zpvw6\") pod \"416aaeab-a8db-432d-ae00-2756d05b8507\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") "
Jul 23 12:42:49.250948 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.250747    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416aaeab-a8db-432d-ae00-2756d05b8507-maas-api-tls\") pod \"416aaeab-a8db-432d-ae00-2756d05b8507\" (UID: \"416aaeab-a8db-432d-ae00-2756d05b8507\") "
Jul 23 12:42:49.251166 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.251135    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/416aaeab-a8db-432d-ae00-2756d05b8507-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "416aaeab-a8db-432d-ae00-2756d05b8507" (UID: "416aaeab-a8db-432d-ae00-2756d05b8507"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:42:49.252916 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.252887    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/416aaeab-a8db-432d-ae00-2756d05b8507-kube-api-access-zpvw6" (OuterVolumeSpecName: "kube-api-access-zpvw6") pod "416aaeab-a8db-432d-ae00-2756d05b8507" (UID: "416aaeab-a8db-432d-ae00-2756d05b8507"). InnerVolumeSpecName "kube-api-access-zpvw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:42:49.253055 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.252959    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/416aaeab-a8db-432d-ae00-2756d05b8507-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "416aaeab-a8db-432d-ae00-2756d05b8507" (UID: "416aaeab-a8db-432d-ae00-2756d05b8507"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:42:49.352058 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.352022    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416aaeab-a8db-432d-ae00-2756d05b8507-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:42:49.352058 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.352053    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpvw6\" (UniqueName: \"kubernetes.io/projected/416aaeab-a8db-432d-ae00-2756d05b8507-kube-api-access-zpvw6\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:42:49.352058 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.352062    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416aaeab-a8db-432d-ae00-2756d05b8507-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:42:49.825562 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.825525    2565 generic.go:356] "Generic (PLEG): container finished" podID="416aaeab-a8db-432d-ae00-2756d05b8507" containerID="144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d" exitCode=0
Jul 23 12:42:49.825748 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.825574    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"
Jul 23 12:42:49.825748 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.825599    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5" event={"ID":"416aaeab-a8db-432d-ae00-2756d05b8507","Type":"ContainerDied","Data":"144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d"}
Jul 23 12:42:49.825748 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.825650    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5" event={"ID":"416aaeab-a8db-432d-ae00-2756d05b8507","Type":"ContainerDied","Data":"f91e0aa6175d54ffd8ee277c1edee162b85007b5aef51e2e1c3e0f490fe4af9d"}
Jul 23 12:42:49.825748 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.825680    2565 scope.go:117] "RemoveContainer" containerID="144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d"
Jul 23 12:42:49.834346 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.834330    2565 scope.go:117] "RemoveContainer" containerID="144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d"
Jul 23 12:42:49.834590 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:42:49.834573    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d\": container with ID starting with 144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d not found: ID does not exist" containerID="144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d"
Jul 23 12:42:49.834647 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.834605    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d"} err="failed to get container status \"144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d\": rpc error: code = NotFound desc = could not find container \"144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d\": container with ID starting with 144320645d0c1593cd532d5356dea4096484f4305c3f920edb20356c179c9e5d not found: ID does not exist"
Jul 23 12:42:49.849384 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.849352    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"]
Jul 23 12:42:49.855014 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:49.854987    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-312a3fbf-55cc886467-29qh5"]
Jul 23 12:42:51.050085 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:51.050049    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="416aaeab-a8db-432d-ae00-2756d05b8507" path="/var/lib/kubelet/pods/416aaeab-a8db-432d-ae00-2756d05b8507/volumes"
Jul 23 12:42:53.676083 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:53.676047    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"]
Jul 23 12:42:53.679221 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:53.679190    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-312a3fbf-qnsdf"]
Jul 23 12:42:55.049776 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:42:55.049738    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8f0d283-fdb2-4bcc-9f15-746f99aaf231" path="/var/lib/kubelet/pods/c8f0d283-fdb2-4bcc-9f15-746f99aaf231/volumes"
Jul 23 12:43:24.053696 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.053616    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"]
Jul 23 12:43:24.054343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.054166    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8f0d283-fdb2-4bcc-9f15-746f99aaf231" containerName="revoke-keys"
Jul 23 12:43:24.054343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.054184    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8f0d283-fdb2-4bcc-9f15-746f99aaf231" containerName="revoke-keys"
Jul 23 12:43:24.054343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.054239    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="416aaeab-a8db-432d-ae00-2756d05b8507" containerName="maas-api"
Jul 23 12:43:24.054343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.054248    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="416aaeab-a8db-432d-ae00-2756d05b8507" containerName="maas-api"
Jul 23 12:43:24.054603 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.054364    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="416aaeab-a8db-432d-ae00-2756d05b8507" containerName="maas-api"
Jul 23 12:43:24.054603 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.054379    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8f0d283-fdb2-4bcc-9f15-746f99aaf231" containerName="revoke-keys"
Jul 23 12:43:24.057729 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.057708    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.060479 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.060450    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e67fae86-serving-cert\""
Jul 23 12:43:24.068140 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.068118    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"]
Jul 23 12:43:24.124775 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.124730    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306c6333-2bc9-4227-8568-5e2b09e7ea96-maas-api-tls\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.124948 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.124817    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306c6333-2bc9-4227-8568-5e2b09e7ea96-openshift-service-ca\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.124948 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.124844    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcgm6\" (UniqueName: \"kubernetes.io/projected/306c6333-2bc9-4227-8568-5e2b09e7ea96-kube-api-access-wcgm6\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.225853 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.225823    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306c6333-2bc9-4227-8568-5e2b09e7ea96-maas-api-tls\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.226061 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.225874    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306c6333-2bc9-4227-8568-5e2b09e7ea96-openshift-service-ca\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.226061 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.225911    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcgm6\" (UniqueName: \"kubernetes.io/projected/306c6333-2bc9-4227-8568-5e2b09e7ea96-kube-api-access-wcgm6\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.226061 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:43:24.225988    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-serving-cert: secret "maas-api-e2e-mt-e67fae86-serving-cert" not found
Jul 23 12:43:24.226241 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:43:24.226070    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306c6333-2bc9-4227-8568-5e2b09e7ea96-maas-api-tls podName:306c6333-2bc9-4227-8568-5e2b09e7ea96 nodeName:}" failed. No retries permitted until 2026-07-23 12:43:24.726048181 +0000 UTC m=+1988.381556968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/306c6333-2bc9-4227-8568-5e2b09e7ea96-maas-api-tls") pod "maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf" (UID: "306c6333-2bc9-4227-8568-5e2b09e7ea96") : secret "maas-api-e2e-mt-e67fae86-serving-cert" not found
Jul 23 12:43:24.226494 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.226475    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306c6333-2bc9-4227-8568-5e2b09e7ea96-openshift-service-ca\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.235285 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.235258    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcgm6\" (UniqueName: \"kubernetes.io/projected/306c6333-2bc9-4227-8568-5e2b09e7ea96-kube-api-access-wcgm6\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.730057 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.730020    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306c6333-2bc9-4227-8568-5e2b09e7ea96-maas-api-tls\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.732487 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.732463    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306c6333-2bc9-4227-8568-5e2b09e7ea96-maas-api-tls\") pod \"maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:24.971749 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:24.971714    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:25.116454 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:25.116420    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"]
Jul 23 12:43:25.117692 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:43:25.117666    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306c6333_2bc9_4227_8568_5e2b09e7ea96.slice/crio-536a4925e00f7d4e4fc9ad897432e7e1c7a8a55416830518b48e3796a95f0792 WatchSource:0}: Error finding container 536a4925e00f7d4e4fc9ad897432e7e1c7a8a55416830518b48e3796a95f0792: Status 404 returned error can't find the container with id 536a4925e00f7d4e4fc9ad897432e7e1c7a8a55416830518b48e3796a95f0792
Jul 23 12:43:25.119002 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:25.118987    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 12:43:26.005664 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:26.005577    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf" event={"ID":"306c6333-2bc9-4227-8568-5e2b09e7ea96","Type":"ContainerStarted","Data":"089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d"}
Jul 23 12:43:26.005664 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:26.005624    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf" event={"ID":"306c6333-2bc9-4227-8568-5e2b09e7ea96","Type":"ContainerStarted","Data":"536a4925e00f7d4e4fc9ad897432e7e1c7a8a55416830518b48e3796a95f0792"}
Jul 23 12:43:26.005881 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:26.005698    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:43:26.023799 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:26.023713    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf" podStartSLOduration=1.5300097240000001 podStartE2EDuration="2.023695098s" totalImagesPullingTime="493.685374ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:43:26.022755368 +0000 UTC m=+1989.678264173" watchObservedRunningTime="2026-07-23 12:43:26.023695098 +0000 UTC m=+1989.679203902"
Jul 23 12:43:32.016603 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:43:32.016572    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:44:07.272449 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:07.272409    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"]
Jul 23 12:44:07.276707 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:07.276685    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"
Jul 23 12:44:07.279923 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:07.279902    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:44:07.283774 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:07.283738    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"]
Jul 23 12:44:07.406186 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:07.406145    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e67fae86-vc92c\" (UID: \"b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"
Jul 23 12:44:07.507105 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:07.507068    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e67fae86-vc92c\" (UID: \"b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"
Jul 23 12:44:07.507782 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:07.507737    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e67fae86-vc92c\" (UID: \"b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"
Jul 23 12:44:07.588745 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:07.588708    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"
Jul 23 12:44:07.734066 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:07.734034    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"]
Jul 23 12:44:07.737783 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:44:07.737728    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a5bb78_f2ab_4c92_ac4b_4ef86353b7b8.slice/crio-fdc1ed03e8616d2644345c28e0f0ab3dd6fd702fe6658877450d0914f79ef99f WatchSource:0}: Error finding container fdc1ed03e8616d2644345c28e0f0ab3dd6fd702fe6658877450d0914f79ef99f: Status 404 returned error can't find the container with id fdc1ed03e8616d2644345c28e0f0ab3dd6fd702fe6658877450d0914f79ef99f
Jul 23 12:44:08.207373 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:08.207289    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c" event={"ID":"b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8","Type":"ContainerStarted","Data":"4a659ef4db42ca873a02b0a9f41fe64a974223052e80565cf820fd19346510ae"}
Jul 23 12:44:08.207373 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:08.207342    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c" event={"ID":"b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8","Type":"ContainerStarted","Data":"fdc1ed03e8616d2644345c28e0f0ab3dd6fd702fe6658877450d0914f79ef99f"}
Jul 23 12:44:08.223447 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:08.223385    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c" podStartSLOduration=1.223362988 podStartE2EDuration="1.223362988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:44:08.222804781 +0000 UTC m=+2031.878313585" watchObservedRunningTime="2026-07-23 12:44:08.223362988 +0000 UTC m=+2031.878871793"
Jul 23 12:44:09.219429 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:09.219394    2565 generic.go:356] "Generic (PLEG): container finished" podID="b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8" containerID="4a659ef4db42ca873a02b0a9f41fe64a974223052e80565cf820fd19346510ae" exitCode=0
Jul 23 12:44:09.219832 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:09.219484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c" event={"ID":"b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8","Type":"ContainerDied","Data":"4a659ef4db42ca873a02b0a9f41fe64a974223052e80565cf820fd19346510ae"}
Jul 23 12:44:10.361037 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:10.361007    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"
Jul 23 12:44:10.535967 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:10.535866    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8-maas-api-service-ca\") pod \"b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8\" (UID: \"b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8\") "
Jul 23 12:44:10.536280 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:10.536256    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8" (UID: "b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:44:10.637550 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:10.637515    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8-maas-api-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:44:11.230313 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:11.230276    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c" event={"ID":"b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8","Type":"ContainerDied","Data":"fdc1ed03e8616d2644345c28e0f0ab3dd6fd702fe6658877450d0914f79ef99f"}
Jul 23 12:44:11.230313 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:11.230301    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"
Jul 23 12:44:11.230522 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:11.230331    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdc1ed03e8616d2644345c28e0f0ab3dd6fd702fe6658877450d0914f79ef99f"
Jul 23 12:44:22.798509 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:22.798467    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"]
Jul 23 12:44:22.799014 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:22.798747    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf" podUID="306c6333-2bc9-4227-8568-5e2b09e7ea96" containerName="maas-api" containerID="cri-o://089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d" gracePeriod=30
Jul 23 12:44:23.056524 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.056502    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:44:23.123711 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.123676    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306c6333-2bc9-4227-8568-5e2b09e7ea96-maas-api-tls\") pod \"306c6333-2bc9-4227-8568-5e2b09e7ea96\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") "
Jul 23 12:44:23.123711 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.123718    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306c6333-2bc9-4227-8568-5e2b09e7ea96-openshift-service-ca\") pod \"306c6333-2bc9-4227-8568-5e2b09e7ea96\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") "
Jul 23 12:44:23.123963 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.123780    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcgm6\" (UniqueName: \"kubernetes.io/projected/306c6333-2bc9-4227-8568-5e2b09e7ea96-kube-api-access-wcgm6\") pod \"306c6333-2bc9-4227-8568-5e2b09e7ea96\" (UID: \"306c6333-2bc9-4227-8568-5e2b09e7ea96\") "
Jul 23 12:44:23.124156 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.124125    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/306c6333-2bc9-4227-8568-5e2b09e7ea96-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "306c6333-2bc9-4227-8568-5e2b09e7ea96" (UID: "306c6333-2bc9-4227-8568-5e2b09e7ea96"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:44:23.125748 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.125731    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/306c6333-2bc9-4227-8568-5e2b09e7ea96-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "306c6333-2bc9-4227-8568-5e2b09e7ea96" (UID: "306c6333-2bc9-4227-8568-5e2b09e7ea96"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:44:23.125882 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.125867    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306c6333-2bc9-4227-8568-5e2b09e7ea96-kube-api-access-wcgm6" (OuterVolumeSpecName: "kube-api-access-wcgm6") pod "306c6333-2bc9-4227-8568-5e2b09e7ea96" (UID: "306c6333-2bc9-4227-8568-5e2b09e7ea96"). InnerVolumeSpecName "kube-api-access-wcgm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:44:23.224930 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.224894    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306c6333-2bc9-4227-8568-5e2b09e7ea96-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:44:23.224930 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.224923    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306c6333-2bc9-4227-8568-5e2b09e7ea96-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:44:23.224930 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.224934    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcgm6\" (UniqueName: \"kubernetes.io/projected/306c6333-2bc9-4227-8568-5e2b09e7ea96-kube-api-access-wcgm6\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:44:23.288878 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.288846    2565 generic.go:356] "Generic (PLEG): container finished" podID="306c6333-2bc9-4227-8568-5e2b09e7ea96" containerID="089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d" exitCode=0
Jul 23 12:44:23.289047 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.288892    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"
Jul 23 12:44:23.289047 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.288930    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf" event={"ID":"306c6333-2bc9-4227-8568-5e2b09e7ea96","Type":"ContainerDied","Data":"089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d"}
Jul 23 12:44:23.289047 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.288988    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf" event={"ID":"306c6333-2bc9-4227-8568-5e2b09e7ea96","Type":"ContainerDied","Data":"536a4925e00f7d4e4fc9ad897432e7e1c7a8a55416830518b48e3796a95f0792"}
Jul 23 12:44:23.289047 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.289019    2565 scope.go:117] "RemoveContainer" containerID="089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d"
Jul 23 12:44:23.298000 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.297983    2565 scope.go:117] "RemoveContainer" containerID="089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d"
Jul 23 12:44:23.298289 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:44:23.298268    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d\": container with ID starting with 089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d not found: ID does not exist" containerID="089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d"
Jul 23 12:44:23.298374 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.298294    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d"} err="failed to get container status \"089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d\": rpc error: code = NotFound desc = could not find container \"089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d\": container with ID starting with 089c37413790558e6ae4939bd25c3b0871fcb7cda232380c87a9b8c954cee69d not found: ID does not exist"
Jul 23 12:44:23.314256 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.314176    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"]
Jul 23 12:44:23.319156 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:23.319130    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e67fae86-7575d7458f-vwdbf"]
Jul 23 12:44:25.049860 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:25.049824    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="306c6333-2bc9-4227-8568-5e2b09e7ea96" path="/var/lib/kubelet/pods/306c6333-2bc9-4227-8568-5e2b09e7ea96/volumes"
Jul 23 12:44:27.363843 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:27.363805    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"]
Jul 23 12:44:27.367835 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:27.367804    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e67fae86-vc92c"]
Jul 23 12:44:29.049708 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:44:29.049674    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8" path="/var/lib/kubelet/pods/b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8/volumes"
Jul 23 12:45:26.709210 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:45:26.709175    2565 scope.go:117] "RemoveContainer" containerID="4df7f9122612d50cabf49285263e2392497ffecfc447fc801d42c1e016fa52a7"
Jul 23 12:45:26.717247 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:45:26.717229    2565 scope.go:117] "RemoveContainer" containerID="caf2f2cffd13151df26436d5c0c8ee8b803d8444a870ca0386eae853c8e02f12"
Jul 23 12:46:18.920173 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.920089    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"]
Jul 23 12:46:18.920702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.920429    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="306c6333-2bc9-4227-8568-5e2b09e7ea96" containerName="maas-api"
Jul 23 12:46:18.920702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.920439    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="306c6333-2bc9-4227-8568-5e2b09e7ea96" containerName="maas-api"
Jul 23 12:46:18.920702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.920459    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8" containerName="revoke-keys"
Jul 23 12:46:18.920702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.920465    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8" containerName="revoke-keys"
Jul 23 12:46:18.920702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.920517    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="306c6333-2bc9-4227-8568-5e2b09e7ea96" containerName="maas-api"
Jul 23 12:46:18.920702 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.920525    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6a5bb78-f2ab-4c92-ac4b-4ef86353b7b8" containerName="revoke-keys"
Jul 23 12:46:18.923645 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.923625    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:18.926388 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.926361    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c652bb-serving-cert\""
Jul 23 12:46:18.935612 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:18.935586    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"]
Jul 23 12:46:19.000812 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.000746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd5fb97a-c712-4440-81c5-c32c330a1da5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.001000 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.000870    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd5fb97a-c712-4440-81c5-c32c330a1da5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.001000 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.000901    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjwjt\" (UniqueName: \"kubernetes.io/projected/cd5fb97a-c712-4440-81c5-c32c330a1da5-kube-api-access-tjwjt\") pod \"maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.101914 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.101883    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd5fb97a-c712-4440-81c5-c32c330a1da5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.102104 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.101948    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd5fb97a-c712-4440-81c5-c32c330a1da5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.102104 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.101965    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjwjt\" (UniqueName: \"kubernetes.io/projected/cd5fb97a-c712-4440-81c5-c32c330a1da5-kube-api-access-tjwjt\") pod \"maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.102617 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.102595    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd5fb97a-c712-4440-81c5-c32c330a1da5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.104334 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.104315    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd5fb97a-c712-4440-81c5-c32c330a1da5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.118451 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.118418    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjwjt\" (UniqueName: \"kubernetes.io/projected/cd5fb97a-c712-4440-81c5-c32c330a1da5-kube-api-access-tjwjt\") pod \"maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.234005 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.233921    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:19.376258 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.376224    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"]
Jul 23 12:46:19.378194 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:46:19.378165    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd5fb97a_c712_4440_81c5_c32c330a1da5.slice/crio-3ef6c17f4753ffbd51fc6394df9d76775dd7035255070983c79c20fb969333e4 WatchSource:0}: Error finding container 3ef6c17f4753ffbd51fc6394df9d76775dd7035255070983c79c20fb969333e4: Status 404 returned error can't find the container with id 3ef6c17f4753ffbd51fc6394df9d76775dd7035255070983c79c20fb969333e4
Jul 23 12:46:19.850105 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:19.850065    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t" event={"ID":"cd5fb97a-c712-4440-81c5-c32c330a1da5","Type":"ContainerStarted","Data":"3ef6c17f4753ffbd51fc6394df9d76775dd7035255070983c79c20fb969333e4"}
Jul 23 12:46:20.857220 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:20.857179    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t" event={"ID":"cd5fb97a-c712-4440-81c5-c32c330a1da5","Type":"ContainerStarted","Data":"39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361"}
Jul 23 12:46:20.857694 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:20.857264    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:20.878906 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:20.878850    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t" podStartSLOduration=2.246828506 podStartE2EDuration="2.878834669s" totalImagesPullingTime="632.006163ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:46:20.876240307 +0000 UTC m=+2164.531749111" watchObservedRunningTime="2026-07-23 12:46:20.878834669 +0000 UTC m=+2164.534343475"
Jul 23 12:46:26.738934 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:26.738898    2565 scope.go:117] "RemoveContainer" containerID="264ed42c75662a7c760ecf2c3f8240809403376f6de826299b1d9824a445f082"
Jul 23 12:46:26.867996 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:26.867969    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:46:27.341679 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.341644    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"]
Jul 23 12:46:27.347133 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.347108    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.349669 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.349649    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-4551ad-serving-cert\""
Jul 23 12:46:27.360097 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.360072    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"]
Jul 23 12:46:27.370202 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.370170    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.371022 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.370975    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-maas-api-tls\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.371149 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.371105    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pngs7\" (UniqueName: \"kubernetes.io/projected/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-kube-api-access-pngs7\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.472208 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.472175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.472379 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.472313    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-maas-api-tls\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.472444 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.472406    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pngs7\" (UniqueName: \"kubernetes.io/projected/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-kube-api-access-pngs7\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.472444 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:46:27.472427    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-serving-cert: secret "maas-api-e2e-inf-b-4551ad-serving-cert" not found
Jul 23 12:46:27.472539 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:46:27.472501    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-maas-api-tls podName:404f9bb8-c92c-4d75-9afb-6b4d982a8f57 nodeName:}" failed. No retries permitted until 2026-07-23 12:46:27.97248175 +0000 UTC m=+2171.627990542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-maas-api-tls") pod "maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v" (UID: "404f9bb8-c92c-4d75-9afb-6b4d982a8f57") : secret "maas-api-e2e-inf-b-4551ad-serving-cert" not found
Jul 23 12:46:27.473018 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.472989    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.481416 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.481389    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pngs7\" (UniqueName: \"kubernetes.io/projected/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-kube-api-access-pngs7\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.976848 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.976813    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-maas-api-tls\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:27.979360 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:27.979328    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-maas-api-tls\") pod \"maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:28.258855 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:28.258755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:28.408343 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:28.408205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"]
Jul 23 12:46:28.411228 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:46:28.411196    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404f9bb8_c92c_4d75_9afb_6b4d982a8f57.slice/crio-53a197027a23d3dfa612438916d0d8568296bcc4a5ba75944084c5f0258f1740 WatchSource:0}: Error finding container 53a197027a23d3dfa612438916d0d8568296bcc4a5ba75944084c5f0258f1740: Status 404 returned error can't find the container with id 53a197027a23d3dfa612438916d0d8568296bcc4a5ba75944084c5f0258f1740
Jul 23 12:46:28.900244 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:28.900214    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v" event={"ID":"404f9bb8-c92c-4d75-9afb-6b4d982a8f57","Type":"ContainerStarted","Data":"53a197027a23d3dfa612438916d0d8568296bcc4a5ba75944084c5f0258f1740"}
Jul 23 12:46:29.907325 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:29.907289    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v" event={"ID":"404f9bb8-c92c-4d75-9afb-6b4d982a8f57","Type":"ContainerStarted","Data":"df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f"}
Jul 23 12:46:29.907783 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:29.907452    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:46:29.926496 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:29.926452    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v" podStartSLOduration=2.465717051 podStartE2EDuration="2.926438168s" totalImagesPullingTime="460.721117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:46:29.92522003 +0000 UTC m=+2173.580728835" watchObservedRunningTime="2026-07-23 12:46:29.926438168 +0000 UTC m=+2173.581946971"
Jul 23 12:46:35.917983 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:46:35.917954    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:48:08.245440 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:08.245346    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"]
Jul 23 12:48:08.248990 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:08.248968    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"
Jul 23 12:48:08.251474 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:08.251444    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f2a794-adb5-43de-9e92-9a0c9f787782-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8\" (UID: \"64f2a794-adb5-43de-9e92-9a0c9f787782\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"
Jul 23 12:48:08.251594 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:08.251515    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:48:08.257835 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:08.257813    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"]
Jul 23 12:48:08.351899 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:08.351854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f2a794-adb5-43de-9e92-9a0c9f787782-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8\" (UID: \"64f2a794-adb5-43de-9e92-9a0c9f787782\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"
Jul 23 12:48:08.352519 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:08.352496    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f2a794-adb5-43de-9e92-9a0c9f787782-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8\" (UID: \"64f2a794-adb5-43de-9e92-9a0c9f787782\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"
Jul 23 12:48:08.561246 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:08.561221    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"
Jul 23 12:48:08.700160 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:08.700072    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"]
Jul 23 12:48:08.702600 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:48:08.702569    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f2a794_adb5_43de_9e92_9a0c9f787782.slice/crio-5024eeb8f65ffc2aa9eca982036eae3d84467489d87858fc220bfb1ca82d58c7 WatchSource:0}: Error finding container 5024eeb8f65ffc2aa9eca982036eae3d84467489d87858fc220bfb1ca82d58c7: Status 404 returned error can't find the container with id 5024eeb8f65ffc2aa9eca982036eae3d84467489d87858fc220bfb1ca82d58c7
Jul 23 12:48:09.371166 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:09.371129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8" event={"ID":"64f2a794-adb5-43de-9e92-9a0c9f787782","Type":"ContainerStarted","Data":"ca85f33988f10127d66fe099d40c2b737f0f52fc4117cff163afdac70ad8707f"}
Jul 23 12:48:09.371563 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:09.371190    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8" event={"ID":"64f2a794-adb5-43de-9e92-9a0c9f787782","Type":"ContainerStarted","Data":"5024eeb8f65ffc2aa9eca982036eae3d84467489d87858fc220bfb1ca82d58c7"}
Jul 23 12:48:09.388439 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:09.388389    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8" podStartSLOduration=1.388373973 podStartE2EDuration="1.388373973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:48:09.387037608 +0000 UTC m=+2273.042546413" watchObservedRunningTime="2026-07-23 12:48:09.388373973 +0000 UTC m=+2273.043882776"
Jul 23 12:48:10.377792 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:10.377732    2565 generic.go:356] "Generic (PLEG): container finished" podID="64f2a794-adb5-43de-9e92-9a0c9f787782" containerID="ca85f33988f10127d66fe099d40c2b737f0f52fc4117cff163afdac70ad8707f" exitCode=0
Jul 23 12:48:10.378216 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:10.377826    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8" event={"ID":"64f2a794-adb5-43de-9e92-9a0c9f787782","Type":"ContainerDied","Data":"ca85f33988f10127d66fe099d40c2b737f0f52fc4117cff163afdac70ad8707f"}
Jul 23 12:48:11.511777 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:11.511739    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"
Jul 23 12:48:11.578310 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:11.578279    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f2a794-adb5-43de-9e92-9a0c9f787782-maas-api-service-ca\") pod \"64f2a794-adb5-43de-9e92-9a0c9f787782\" (UID: \"64f2a794-adb5-43de-9e92-9a0c9f787782\") "
Jul 23 12:48:11.578651 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:11.578630    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64f2a794-adb5-43de-9e92-9a0c9f787782-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64f2a794-adb5-43de-9e92-9a0c9f787782" (UID: "64f2a794-adb5-43de-9e92-9a0c9f787782"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:48:11.679223 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:11.679144    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f2a794-adb5-43de-9e92-9a0c9f787782-maas-api-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:48:12.388225 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:12.388198    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"
Jul 23 12:48:12.388412 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:12.388189    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8" event={"ID":"64f2a794-adb5-43de-9e92-9a0c9f787782","Type":"ContainerDied","Data":"5024eeb8f65ffc2aa9eca982036eae3d84467489d87858fc220bfb1ca82d58c7"}
Jul 23 12:48:12.388412 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:12.388318    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5024eeb8f65ffc2aa9eca982036eae3d84467489d87858fc220bfb1ca82d58c7"
Jul 23 12:48:18.713208 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:18.713170    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"]
Jul 23 12:48:18.713594 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:18.713449    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t" podUID="cd5fb97a-c712-4440-81c5-c32c330a1da5" containerName="maas-api" containerID="cri-o://39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361" gracePeriod=30
Jul 23 12:48:18.967640 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:18.967579    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:48:19.040823 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.040785    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjwjt\" (UniqueName: \"kubernetes.io/projected/cd5fb97a-c712-4440-81c5-c32c330a1da5-kube-api-access-tjwjt\") pod \"cd5fb97a-c712-4440-81c5-c32c330a1da5\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") "
Jul 23 12:48:19.041018 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.040845    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd5fb97a-c712-4440-81c5-c32c330a1da5-maas-api-tls\") pod \"cd5fb97a-c712-4440-81c5-c32c330a1da5\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") "
Jul 23 12:48:19.041018 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.040915    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd5fb97a-c712-4440-81c5-c32c330a1da5-openshift-service-ca\") pod \"cd5fb97a-c712-4440-81c5-c32c330a1da5\" (UID: \"cd5fb97a-c712-4440-81c5-c32c330a1da5\") "
Jul 23 12:48:19.041507 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.041479    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd5fb97a-c712-4440-81c5-c32c330a1da5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cd5fb97a-c712-4440-81c5-c32c330a1da5" (UID: "cd5fb97a-c712-4440-81c5-c32c330a1da5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:48:19.043128 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.043097    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd5fb97a-c712-4440-81c5-c32c330a1da5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cd5fb97a-c712-4440-81c5-c32c330a1da5" (UID: "cd5fb97a-c712-4440-81c5-c32c330a1da5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:48:19.043128 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.043110    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd5fb97a-c712-4440-81c5-c32c330a1da5-kube-api-access-tjwjt" (OuterVolumeSpecName: "kube-api-access-tjwjt") pod "cd5fb97a-c712-4440-81c5-c32c330a1da5" (UID: "cd5fb97a-c712-4440-81c5-c32c330a1da5"). InnerVolumeSpecName "kube-api-access-tjwjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:48:19.142037 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.142003    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjwjt\" (UniqueName: \"kubernetes.io/projected/cd5fb97a-c712-4440-81c5-c32c330a1da5-kube-api-access-tjwjt\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:48:19.142037 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.142029    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd5fb97a-c712-4440-81c5-c32c330a1da5-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:48:19.142037 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.142042    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd5fb97a-c712-4440-81c5-c32c330a1da5-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:48:19.422249 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.422217    2565 generic.go:356] "Generic (PLEG): container finished" podID="cd5fb97a-c712-4440-81c5-c32c330a1da5" containerID="39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361" exitCode=0
Jul 23 12:48:19.422461 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.422260    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t" event={"ID":"cd5fb97a-c712-4440-81c5-c32c330a1da5","Type":"ContainerDied","Data":"39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361"}
Jul 23 12:48:19.422461 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.422262    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"
Jul 23 12:48:19.422461 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.422308    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t" event={"ID":"cd5fb97a-c712-4440-81c5-c32c330a1da5","Type":"ContainerDied","Data":"3ef6c17f4753ffbd51fc6394df9d76775dd7035255070983c79c20fb969333e4"}
Jul 23 12:48:19.422461 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.422329    2565 scope.go:117] "RemoveContainer" containerID="39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361"
Jul 23 12:48:19.431484 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.431465    2565 scope.go:117] "RemoveContainer" containerID="39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361"
Jul 23 12:48:19.431803 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:48:19.431780    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361\": container with ID starting with 39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361 not found: ID does not exist" containerID="39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361"
Jul 23 12:48:19.431889 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.431814    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361"} err="failed to get container status \"39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361\": rpc error: code = NotFound desc = could not find container \"39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361\": container with ID starting with 39c95bd020e945922721958804f793cec61752d4ba6249a553fc24f824791361 not found: ID does not exist"
Jul 23 12:48:19.442623 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.442599    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"]
Jul 23 12:48:19.446442 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:19.446413    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c652bb-5779c64c55-74g8t"]
Jul 23 12:48:21.049721 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:21.049680    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd5fb97a-c712-4440-81c5-c32c330a1da5" path="/var/lib/kubelet/pods/cd5fb97a-c712-4440-81c5-c32c330a1da5/volumes"
Jul 23 12:48:23.374572 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:23.374539    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"]
Jul 23 12:48:23.379004 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:23.378979    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c652bb-7j2t8"]
Jul 23 12:48:25.049768 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:25.049729    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64f2a794-adb5-43de-9e92-9a0c9f787782" path="/var/lib/kubelet/pods/64f2a794-adb5-43de-9e92-9a0c9f787782/volumes"
Jul 23 12:48:31.026264 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.026232    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"]
Jul 23 12:48:31.026639 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.026607    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64f2a794-adb5-43de-9e92-9a0c9f787782" containerName="revoke-keys"
Jul 23 12:48:31.026639 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.026618    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="64f2a794-adb5-43de-9e92-9a0c9f787782" containerName="revoke-keys"
Jul 23 12:48:31.026729 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.026642    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd5fb97a-c712-4440-81c5-c32c330a1da5" containerName="maas-api"
Jul 23 12:48:31.026729 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.026648    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd5fb97a-c712-4440-81c5-c32c330a1da5" containerName="maas-api"
Jul 23 12:48:31.026729 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.026698    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd5fb97a-c712-4440-81c5-c32c330a1da5" containerName="maas-api"
Jul 23 12:48:31.026729 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.026705    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="64f2a794-adb5-43de-9e92-9a0c9f787782" containerName="revoke-keys"
Jul 23 12:48:31.030814 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.030795    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"
Jul 23 12:48:31.033593 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.033571    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:48:31.037655 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.037626    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"]
Jul 23 12:48:31.142996 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.142953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68874666-9ecb-42d2-965a-28134f723f50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl\" (UID: \"68874666-9ecb-42d2-965a-28134f723f50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"
Jul 23 12:48:31.243496 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.243458    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68874666-9ecb-42d2-965a-28134f723f50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl\" (UID: \"68874666-9ecb-42d2-965a-28134f723f50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"
Jul 23 12:48:31.244187 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.244160    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68874666-9ecb-42d2-965a-28134f723f50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl\" (UID: \"68874666-9ecb-42d2-965a-28134f723f50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"
Jul 23 12:48:31.342476 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.342445    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"
Jul 23 12:48:31.481095 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:31.481053    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"]
Jul 23 12:48:31.483512 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:48:31.483486    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68874666_9ecb_42d2_965a_28134f723f50.slice/crio-a5f3aed8a25d252f35e6973b5fb09bdd1efd17418cdd3e3573b667b3b3adcb04 WatchSource:0}: Error finding container a5f3aed8a25d252f35e6973b5fb09bdd1efd17418cdd3e3573b667b3b3adcb04: Status 404 returned error can't find the container with id a5f3aed8a25d252f35e6973b5fb09bdd1efd17418cdd3e3573b667b3b3adcb04
Jul 23 12:48:32.493778 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:32.493679    2565 generic.go:356] "Generic (PLEG): container finished" podID="68874666-9ecb-42d2-965a-28134f723f50" containerID="08879d754b274f87c5d84289b42340d90f006f0074f60440b19bf689cb9f3f10" exitCode=0
Jul 23 12:48:32.494121 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:32.493783    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl" event={"ID":"68874666-9ecb-42d2-965a-28134f723f50","Type":"ContainerDied","Data":"08879d754b274f87c5d84289b42340d90f006f0074f60440b19bf689cb9f3f10"}
Jul 23 12:48:32.494121 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:32.493828    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl" event={"ID":"68874666-9ecb-42d2-965a-28134f723f50","Type":"ContainerStarted","Data":"a5f3aed8a25d252f35e6973b5fb09bdd1efd17418cdd3e3573b667b3b3adcb04"}
Jul 23 12:48:33.630118 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:33.630089    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"
Jul 23 12:48:33.769481 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:33.769399    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68874666-9ecb-42d2-965a-28134f723f50-maas-api-service-ca\") pod \"68874666-9ecb-42d2-965a-28134f723f50\" (UID: \"68874666-9ecb-42d2-965a-28134f723f50\") "
Jul 23 12:48:33.769781 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:33.769738    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68874666-9ecb-42d2-965a-28134f723f50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "68874666-9ecb-42d2-965a-28134f723f50" (UID: "68874666-9ecb-42d2-965a-28134f723f50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:48:33.870831 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:33.870792    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68874666-9ecb-42d2-965a-28134f723f50-maas-api-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:48:34.504431 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:34.504402    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"
Jul 23 12:48:34.504611 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:34.504451    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl" event={"ID":"68874666-9ecb-42d2-965a-28134f723f50","Type":"ContainerDied","Data":"a5f3aed8a25d252f35e6973b5fb09bdd1efd17418cdd3e3573b667b3b3adcb04"}
Jul 23 12:48:34.504611 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:34.504502    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5f3aed8a25d252f35e6973b5fb09bdd1efd17418cdd3e3573b667b3b3adcb04"
Jul 23 12:48:41.602667 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:41.602636    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"]
Jul 23 12:48:41.603287 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:41.602898    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v" podUID="404f9bb8-c92c-4d75-9afb-6b4d982a8f57" containerName="maas-api" containerID="cri-o://df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f" gracePeriod=30
Jul 23 12:48:41.851297 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:41.851269    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:48:41.941384 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:41.941289    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pngs7\" (UniqueName: \"kubernetes.io/projected/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-kube-api-access-pngs7\") pod \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") "
Jul 23 12:48:41.941384 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:41.941325    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-maas-api-tls\") pod \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") "
Jul 23 12:48:41.941384 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:41.941361    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-openshift-service-ca\") pod \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\" (UID: \"404f9bb8-c92c-4d75-9afb-6b4d982a8f57\") "
Jul 23 12:48:41.941735 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:41.941711    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "404f9bb8-c92c-4d75-9afb-6b4d982a8f57" (UID: "404f9bb8-c92c-4d75-9afb-6b4d982a8f57"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:48:41.943581 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:41.943552    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "404f9bb8-c92c-4d75-9afb-6b4d982a8f57" (UID: "404f9bb8-c92c-4d75-9afb-6b4d982a8f57"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:48:41.943721 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:41.943600    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-kube-api-access-pngs7" (OuterVolumeSpecName: "kube-api-access-pngs7") pod "404f9bb8-c92c-4d75-9afb-6b4d982a8f57" (UID: "404f9bb8-c92c-4d75-9afb-6b4d982a8f57"). InnerVolumeSpecName "kube-api-access-pngs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:48:42.042080 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.042039    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pngs7\" (UniqueName: \"kubernetes.io/projected/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-kube-api-access-pngs7\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:48:42.042080 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.042080    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:48:42.042297 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.042096    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/404f9bb8-c92c-4d75-9afb-6b4d982a8f57-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:48:42.543838 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.543800    2565 generic.go:356] "Generic (PLEG): container finished" podID="404f9bb8-c92c-4d75-9afb-6b4d982a8f57" containerID="df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f" exitCode=0
Jul 23 12:48:42.544006 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.543864    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"
Jul 23 12:48:42.544006 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.543885    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v" event={"ID":"404f9bb8-c92c-4d75-9afb-6b4d982a8f57","Type":"ContainerDied","Data":"df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f"}
Jul 23 12:48:42.544006 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.543957    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v" event={"ID":"404f9bb8-c92c-4d75-9afb-6b4d982a8f57","Type":"ContainerDied","Data":"53a197027a23d3dfa612438916d0d8568296bcc4a5ba75944084c5f0258f1740"}
Jul 23 12:48:42.544006 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.543980    2565 scope.go:117] "RemoveContainer" containerID="df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f"
Jul 23 12:48:42.553091 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.553074    2565 scope.go:117] "RemoveContainer" containerID="df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f"
Jul 23 12:48:42.553357 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:48:42.553339    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f\": container with ID starting with df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f not found: ID does not exist" containerID="df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f"
Jul 23 12:48:42.553409 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.553366    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f"} err="failed to get container status \"df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f\": rpc error: code = NotFound desc = could not find container \"df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f\": container with ID starting with df2490dbf19bf573beb2be06c10086e19487b1046b9bf66c2e52161ace21cc8f not found: ID does not exist"
Jul 23 12:48:42.567219 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.567191    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"]
Jul 23 12:48:42.571644 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:42.571617    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4551ad-74898bfb54-9tg2v"]
Jul 23 12:48:43.050528 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:43.050491    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="404f9bb8-c92c-4d75-9afb-6b4d982a8f57" path="/var/lib/kubelet/pods/404f9bb8-c92c-4d75-9afb-6b4d982a8f57/volumes"
Jul 23 12:48:46.110689 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:46.110653    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"]
Jul 23 12:48:46.113275 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:46.113249    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4551ad-b5wbl"]
Jul 23 12:48:47.050131 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:47.050097    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68874666-9ecb-42d2-965a-28134f723f50" path="/var/lib/kubelet/pods/68874666-9ecb-42d2-965a-28134f723f50/volumes"
Jul 23 12:48:56.688039 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.688003    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"]
Jul 23 12:48:56.688479 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.688403    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="404f9bb8-c92c-4d75-9afb-6b4d982a8f57" containerName="maas-api"
Jul 23 12:48:56.688479 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.688417    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="404f9bb8-c92c-4d75-9afb-6b4d982a8f57" containerName="maas-api"
Jul 23 12:48:56.688479 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.688430    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68874666-9ecb-42d2-965a-28134f723f50" containerName="revoke-keys"
Jul 23 12:48:56.688479 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.688439    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="68874666-9ecb-42d2-965a-28134f723f50" containerName="revoke-keys"
Jul 23 12:48:56.688653 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.688502    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="404f9bb8-c92c-4d75-9afb-6b4d982a8f57" containerName="maas-api"
Jul 23 12:48:56.688653 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.688514    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="68874666-9ecb-42d2-965a-28134f723f50" containerName="revoke-keys"
Jul 23 12:48:56.691435 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.691418    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:56.694276 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.694253    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-7dfea2-serving-cert\""
Jul 23 12:48:56.702701 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.702668    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"]
Jul 23 12:48:56.875810 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.875755    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zq6j\" (UniqueName: \"kubernetes.io/projected/406d8d6a-dd82-4ad6-b32b-58251566f32c-kube-api-access-4zq6j\") pod \"maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:56.876004 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.875845    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/406d8d6a-dd82-4ad6-b32b-58251566f32c-maas-api-tls\") pod \"maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:56.876004 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.875889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/406d8d6a-dd82-4ad6-b32b-58251566f32c-openshift-service-ca\") pod \"maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:56.976889 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.976794    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zq6j\" (UniqueName: \"kubernetes.io/projected/406d8d6a-dd82-4ad6-b32b-58251566f32c-kube-api-access-4zq6j\") pod \"maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:56.976889 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.976848    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/406d8d6a-dd82-4ad6-b32b-58251566f32c-maas-api-tls\") pod \"maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:56.976889 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.976880    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/406d8d6a-dd82-4ad6-b32b-58251566f32c-openshift-service-ca\") pod \"maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:56.977477 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.977454    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/406d8d6a-dd82-4ad6-b32b-58251566f32c-openshift-service-ca\") pod \"maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:56.979288 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.979266    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/406d8d6a-dd82-4ad6-b32b-58251566f32c-maas-api-tls\") pod \"maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:56.992552 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:56.992529    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zq6j\" (UniqueName: \"kubernetes.io/projected/406d8d6a-dd82-4ad6-b32b-58251566f32c-kube-api-access-4zq6j\") pod \"maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:57.003409 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:57.003378    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:57.149690 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:57.149662    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"]
Jul 23 12:48:57.152554 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:48:57.152522    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod406d8d6a_dd82_4ad6_b32b_58251566f32c.slice/crio-71b498121bf43680ffd087b82e56f70a6e3990846a5bd4986604a22a1dd67283 WatchSource:0}: Error finding container 71b498121bf43680ffd087b82e56f70a6e3990846a5bd4986604a22a1dd67283: Status 404 returned error can't find the container with id 71b498121bf43680ffd087b82e56f70a6e3990846a5bd4986604a22a1dd67283
Jul 23 12:48:57.153832 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:57.153811    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 12:48:57.617775 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:57.617728    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77" event={"ID":"406d8d6a-dd82-4ad6-b32b-58251566f32c","Type":"ContainerStarted","Data":"71b498121bf43680ffd087b82e56f70a6e3990846a5bd4986604a22a1dd67283"}
Jul 23 12:48:58.624071 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:58.624037    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77" event={"ID":"406d8d6a-dd82-4ad6-b32b-58251566f32c","Type":"ContainerStarted","Data":"182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe"}
Jul 23 12:48:58.624440 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:58.624159    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:48:58.643143 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:48:58.643096    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77" podStartSLOduration=1.853161128 podStartE2EDuration="2.643078431s" totalImagesPullingTime="789.917303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:48:58.642412786 +0000 UTC m=+2322.297921591" watchObservedRunningTime="2026-07-23 12:48:58.643078431 +0000 UTC m=+2322.298587234"
Jul 23 12:49:04.633776 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:04.633729    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:49:05.255712 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.255667    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"]
Jul 23 12:49:05.260018 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.259996    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.263109 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.263088    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-52006d-serving-cert\""
Jul 23 12:49:05.271658 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.271634    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"]
Jul 23 12:49:05.338613 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.338572    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b40cafa-09e8-454b-a52d-3873b6c42296-openshift-service-ca\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.338813 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.338630    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skcqb\" (UniqueName: \"kubernetes.io/projected/4b40cafa-09e8-454b-a52d-3873b6c42296-kube-api-access-skcqb\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.338813 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.338658    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b40cafa-09e8-454b-a52d-3873b6c42296-maas-api-tls\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.439394 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.439354    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b40cafa-09e8-454b-a52d-3873b6c42296-openshift-service-ca\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.439394 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.439400    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skcqb\" (UniqueName: \"kubernetes.io/projected/4b40cafa-09e8-454b-a52d-3873b6c42296-kube-api-access-skcqb\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.439660 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.439427    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b40cafa-09e8-454b-a52d-3873b6c42296-maas-api-tls\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.439660 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:49:05.439578    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-serving-cert: secret "maas-api-e2e-api-b-52006d-serving-cert" not found
Jul 23 12:49:05.439786 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:49:05.439675    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b40cafa-09e8-454b-a52d-3873b6c42296-maas-api-tls podName:4b40cafa-09e8-454b-a52d-3873b6c42296 nodeName:}" failed. No retries permitted until 2026-07-23 12:49:05.939655689 +0000 UTC m=+2329.595164484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b40cafa-09e8-454b-a52d-3873b6c42296-maas-api-tls") pod "maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt" (UID: "4b40cafa-09e8-454b-a52d-3873b6c42296") : secret "maas-api-e2e-api-b-52006d-serving-cert" not found
Jul 23 12:49:05.440010 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.439991    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b40cafa-09e8-454b-a52d-3873b6c42296-openshift-service-ca\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.449605 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.449579    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skcqb\" (UniqueName: \"kubernetes.io/projected/4b40cafa-09e8-454b-a52d-3873b6c42296-kube-api-access-skcqb\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.943736 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.943701    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b40cafa-09e8-454b-a52d-3873b6c42296-maas-api-tls\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:05.946119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:05.946089    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b40cafa-09e8-454b-a52d-3873b6c42296-maas-api-tls\") pod \"maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:06.172853 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:06.172822    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:06.311402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:06.311378    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"]
Jul 23 12:49:06.313931 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:49:06.313899    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b40cafa_09e8_454b_a52d_3873b6c42296.slice/crio-4848fef357cea07b7e370d0d9768b157c7057dac92e4e6abfe6148db4b96dcb9 WatchSource:0}: Error finding container 4848fef357cea07b7e370d0d9768b157c7057dac92e4e6abfe6148db4b96dcb9: Status 404 returned error can't find the container with id 4848fef357cea07b7e370d0d9768b157c7057dac92e4e6abfe6148db4b96dcb9
Jul 23 12:49:06.669520 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:06.669484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt" event={"ID":"4b40cafa-09e8-454b-a52d-3873b6c42296","Type":"ContainerStarted","Data":"4848fef357cea07b7e370d0d9768b157c7057dac92e4e6abfe6148db4b96dcb9"}
Jul 23 12:49:07.676376 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:07.676301    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt" event={"ID":"4b40cafa-09e8-454b-a52d-3873b6c42296","Type":"ContainerStarted","Data":"bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e"}
Jul 23 12:49:07.676739 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:07.676396    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:07.700614 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:07.700559    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt" podStartSLOduration=2.327197833 podStartE2EDuration="2.700543512s" totalImagesPullingTime="373.345679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:49:07.698830461 +0000 UTC m=+2331.354339266" watchObservedRunningTime="2026-07-23 12:49:07.700543512 +0000 UTC m=+2331.356052315"
Jul 23 12:49:13.686468 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:13.686443    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:49:17.979371 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:17.979331    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"]
Jul 23 12:49:17.983063 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:17.983042    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"
Jul 23 12:49:17.985842 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:17.985821    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:49:17.992292 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:17.992272    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"]
Jul 23 12:49:18.038645 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:18.038606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3e3d47-47d4-40aa-ade1-c18a4ee9885f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr\" (UID: \"ba3e3d47-47d4-40aa-ade1-c18a4ee9885f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"
Jul 23 12:49:18.139045 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:18.139002    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3e3d47-47d4-40aa-ade1-c18a4ee9885f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr\" (UID: \"ba3e3d47-47d4-40aa-ade1-c18a4ee9885f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"
Jul 23 12:49:18.139689 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:18.139666    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3e3d47-47d4-40aa-ade1-c18a4ee9885f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr\" (UID: \"ba3e3d47-47d4-40aa-ade1-c18a4ee9885f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"
Jul 23 12:49:18.295036 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:18.294949    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"
Jul 23 12:49:18.430593 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:18.430529    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"]
Jul 23 12:49:18.433029 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:49:18.433004    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba3e3d47_47d4_40aa_ade1_c18a4ee9885f.slice/crio-2eab39204a5a4c7067d1570d2254da3d13a7763a895ece62efa422aaf4f1eecb WatchSource:0}: Error finding container 2eab39204a5a4c7067d1570d2254da3d13a7763a895ece62efa422aaf4f1eecb: Status 404 returned error can't find the container with id 2eab39204a5a4c7067d1570d2254da3d13a7763a895ece62efa422aaf4f1eecb
Jul 23 12:49:18.732678 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:18.732644    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr" event={"ID":"ba3e3d47-47d4-40aa-ade1-c18a4ee9885f","Type":"ContainerStarted","Data":"2eab39204a5a4c7067d1570d2254da3d13a7763a895ece62efa422aaf4f1eecb"}
Jul 23 12:49:19.740099 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:19.740058    2565 generic.go:356] "Generic (PLEG): container finished" podID="ba3e3d47-47d4-40aa-ade1-c18a4ee9885f" containerID="3fca467526f7e18494825fb53aa07059c2dd08cd24ad75812951f91553563ced" exitCode=0
Jul 23 12:49:19.740509 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:19.740142    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr" event={"ID":"ba3e3d47-47d4-40aa-ade1-c18a4ee9885f","Type":"ContainerDied","Data":"3fca467526f7e18494825fb53aa07059c2dd08cd24ad75812951f91553563ced"}
Jul 23 12:49:20.893603 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:20.893581    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"
Jul 23 12:49:20.963270 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:20.963239    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3e3d47-47d4-40aa-ade1-c18a4ee9885f-maas-api-service-ca\") pod \"ba3e3d47-47d4-40aa-ade1-c18a4ee9885f\" (UID: \"ba3e3d47-47d4-40aa-ade1-c18a4ee9885f\") "
Jul 23 12:49:20.963594 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:20.963571    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba3e3d47-47d4-40aa-ade1-c18a4ee9885f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba3e3d47-47d4-40aa-ade1-c18a4ee9885f" (UID: "ba3e3d47-47d4-40aa-ade1-c18a4ee9885f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:49:21.064505 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:21.064472    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3e3d47-47d4-40aa-ade1-c18a4ee9885f-maas-api-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:49:21.752784 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:21.752735    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"
Jul 23 12:49:21.752963 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:21.752791    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr" event={"ID":"ba3e3d47-47d4-40aa-ade1-c18a4ee9885f","Type":"ContainerDied","Data":"2eab39204a5a4c7067d1570d2254da3d13a7763a895ece62efa422aaf4f1eecb"}
Jul 23 12:49:21.752963 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:21.752840    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2eab39204a5a4c7067d1570d2254da3d13a7763a895ece62efa422aaf4f1eecb"
Jul 23 12:49:26.776650 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:26.776612    2565 scope.go:117] "RemoveContainer" containerID="c8881aa5b2948163c6eab39e37b0a6623654c45ce2dbf4c2fcc5afbe7d21adeb"
Jul 23 12:49:33.943179 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:33.943145    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"]
Jul 23 12:49:33.943645 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:33.943358    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77" podUID="406d8d6a-dd82-4ad6-b32b-58251566f32c" containerName="maas-api" containerID="cri-o://182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe" gracePeriod=30
Jul 23 12:49:34.196201 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.196139    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:49:34.282867 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.282833    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zq6j\" (UniqueName: \"kubernetes.io/projected/406d8d6a-dd82-4ad6-b32b-58251566f32c-kube-api-access-4zq6j\") pod \"406d8d6a-dd82-4ad6-b32b-58251566f32c\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") "
Jul 23 12:49:34.283060 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.282908    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/406d8d6a-dd82-4ad6-b32b-58251566f32c-maas-api-tls\") pod \"406d8d6a-dd82-4ad6-b32b-58251566f32c\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") "
Jul 23 12:49:34.283060 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.282948    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/406d8d6a-dd82-4ad6-b32b-58251566f32c-openshift-service-ca\") pod \"406d8d6a-dd82-4ad6-b32b-58251566f32c\" (UID: \"406d8d6a-dd82-4ad6-b32b-58251566f32c\") "
Jul 23 12:49:34.283346 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.283315    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/406d8d6a-dd82-4ad6-b32b-58251566f32c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "406d8d6a-dd82-4ad6-b32b-58251566f32c" (UID: "406d8d6a-dd82-4ad6-b32b-58251566f32c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:49:34.284906 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.284880    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/406d8d6a-dd82-4ad6-b32b-58251566f32c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "406d8d6a-dd82-4ad6-b32b-58251566f32c" (UID: "406d8d6a-dd82-4ad6-b32b-58251566f32c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:49:34.284999 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.284905    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406d8d6a-dd82-4ad6-b32b-58251566f32c-kube-api-access-4zq6j" (OuterVolumeSpecName: "kube-api-access-4zq6j") pod "406d8d6a-dd82-4ad6-b32b-58251566f32c" (UID: "406d8d6a-dd82-4ad6-b32b-58251566f32c"). InnerVolumeSpecName "kube-api-access-4zq6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:49:34.384080 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.384050    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zq6j\" (UniqueName: \"kubernetes.io/projected/406d8d6a-dd82-4ad6-b32b-58251566f32c-kube-api-access-4zq6j\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:49:34.384080 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.384076    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/406d8d6a-dd82-4ad6-b32b-58251566f32c-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:49:34.384080 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.384087    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/406d8d6a-dd82-4ad6-b32b-58251566f32c-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:49:34.817190 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.817157    2565 generic.go:356] "Generic (PLEG): container finished" podID="406d8d6a-dd82-4ad6-b32b-58251566f32c" containerID="182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe" exitCode=0
Jul 23 12:49:34.817359 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.817197    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"
Jul 23 12:49:34.817359 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.817241    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77" event={"ID":"406d8d6a-dd82-4ad6-b32b-58251566f32c","Type":"ContainerDied","Data":"182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe"}
Jul 23 12:49:34.817359 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.817287    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77" event={"ID":"406d8d6a-dd82-4ad6-b32b-58251566f32c","Type":"ContainerDied","Data":"71b498121bf43680ffd087b82e56f70a6e3990846a5bd4986604a22a1dd67283"}
Jul 23 12:49:34.817359 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.817311    2565 scope.go:117] "RemoveContainer" containerID="182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe"
Jul 23 12:49:34.826014 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.826001    2565 scope.go:117] "RemoveContainer" containerID="182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe"
Jul 23 12:49:34.826285 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:49:34.826268    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe\": container with ID starting with 182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe not found: ID does not exist" containerID="182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe"
Jul 23 12:49:34.826329 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.826293    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe"} err="failed to get container status \"182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe\": rpc error: code = NotFound desc = could not find container \"182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe\": container with ID starting with 182c02741b0fd22335971f2ba0da502cca57ae83336868b0fff65131e0fc2fbe not found: ID does not exist"
Jul 23 12:49:34.839664 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.839642    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"]
Jul 23 12:49:34.846309 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:34.846284    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7dfea2-b4d7d4949-mzw77"]
Jul 23 12:49:35.050396 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:35.050365    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="406d8d6a-dd82-4ad6-b32b-58251566f32c" path="/var/lib/kubelet/pods/406d8d6a-dd82-4ad6-b32b-58251566f32c/volumes"
Jul 23 12:49:38.075721 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:38.075690    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"]
Jul 23 12:49:38.078564 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:38.078542    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7dfea2-2vnfr"]
Jul 23 12:49:39.049163 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:39.049131    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba3e3d47-47d4-40aa-ade1-c18a4ee9885f" path="/var/lib/kubelet/pods/ba3e3d47-47d4-40aa-ade1-c18a4ee9885f/volumes"
Jul 23 12:49:45.497545 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.497510    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"]
Jul 23 12:49:45.498032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.497907    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba3e3d47-47d4-40aa-ade1-c18a4ee9885f" containerName="revoke-keys"
Jul 23 12:49:45.498032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.497919    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba3e3d47-47d4-40aa-ade1-c18a4ee9885f" containerName="revoke-keys"
Jul 23 12:49:45.498032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.497934    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406d8d6a-dd82-4ad6-b32b-58251566f32c" containerName="maas-api"
Jul 23 12:49:45.498032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.497939    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="406d8d6a-dd82-4ad6-b32b-58251566f32c" containerName="maas-api"
Jul 23 12:49:45.498032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.497999    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="406d8d6a-dd82-4ad6-b32b-58251566f32c" containerName="maas-api"
Jul 23 12:49:45.498032 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.498008    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba3e3d47-47d4-40aa-ade1-c18a4ee9885f" containerName="revoke-keys"
Jul 23 12:49:45.502247 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.502228    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"
Jul 23 12:49:45.504996 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.504975    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:49:45.510651 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.510624    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"]
Jul 23 12:49:45.582275 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.582244    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c208711-96c8-4f6d-b96c-4198cfa62f22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-52006d-x5jpq\" (UID: \"8c208711-96c8-4f6d-b96c-4198cfa62f22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"
Jul 23 12:49:45.683833 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.683803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c208711-96c8-4f6d-b96c-4198cfa62f22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-52006d-x5jpq\" (UID: \"8c208711-96c8-4f6d-b96c-4198cfa62f22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"
Jul 23 12:49:45.684437 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.684416    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c208711-96c8-4f6d-b96c-4198cfa62f22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-52006d-x5jpq\" (UID: \"8c208711-96c8-4f6d-b96c-4198cfa62f22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"
Jul 23 12:49:45.814675 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.814642    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"
Jul 23 12:49:45.955256 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:45.955229    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"]
Jul 23 12:49:45.957590 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:49:45.957564    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c208711_96c8_4f6d_b96c_4198cfa62f22.slice/crio-6f16dc03a82c844a55a19efebb66a108282b9fe733c1cafc2cd1601b77ccd0ea WatchSource:0}: Error finding container 6f16dc03a82c844a55a19efebb66a108282b9fe733c1cafc2cd1601b77ccd0ea: Status 404 returned error can't find the container with id 6f16dc03a82c844a55a19efebb66a108282b9fe733c1cafc2cd1601b77ccd0ea
Jul 23 12:49:46.881191 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:46.881157    2565 generic.go:356] "Generic (PLEG): container finished" podID="8c208711-96c8-4f6d-b96c-4198cfa62f22" containerID="751cc0d5c3f0fc1ca9e77534ec6ae5c02c80d7fe5bc9f88625bd609202499633" exitCode=0
Jul 23 12:49:46.881585 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:46.881248    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq" event={"ID":"8c208711-96c8-4f6d-b96c-4198cfa62f22","Type":"ContainerDied","Data":"751cc0d5c3f0fc1ca9e77534ec6ae5c02c80d7fe5bc9f88625bd609202499633"}
Jul 23 12:49:46.881585 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:46.881336    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq" event={"ID":"8c208711-96c8-4f6d-b96c-4198cfa62f22","Type":"ContainerStarted","Data":"6f16dc03a82c844a55a19efebb66a108282b9fe733c1cafc2cd1601b77ccd0ea"}
Jul 23 12:49:48.017939 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:48.017915    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"
Jul 23 12:49:48.106128 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:48.106095    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c208711-96c8-4f6d-b96c-4198cfa62f22-maas-api-service-ca\") pod \"8c208711-96c8-4f6d-b96c-4198cfa62f22\" (UID: \"8c208711-96c8-4f6d-b96c-4198cfa62f22\") "
Jul 23 12:49:48.106493 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:48.106464    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c208711-96c8-4f6d-b96c-4198cfa62f22-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c208711-96c8-4f6d-b96c-4198cfa62f22" (UID: "8c208711-96c8-4f6d-b96c-4198cfa62f22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:49:48.207213 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:48.207123    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c208711-96c8-4f6d-b96c-4198cfa62f22-maas-api-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:49:48.892932 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:48.892901    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"
Jul 23 12:49:48.892932 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:48.892930    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq" event={"ID":"8c208711-96c8-4f6d-b96c-4198cfa62f22","Type":"ContainerDied","Data":"6f16dc03a82c844a55a19efebb66a108282b9fe733c1cafc2cd1601b77ccd0ea"}
Jul 23 12:49:48.893152 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:49:48.892969    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f16dc03a82c844a55a19efebb66a108282b9fe733c1cafc2cd1601b77ccd0ea"
Jul 23 12:50:00.988056 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:00.988009    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"]
Jul 23 12:50:00.988593 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:00.988270    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt" podUID="4b40cafa-09e8-454b-a52d-3873b6c42296" containerName="maas-api" containerID="cri-o://bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e" gracePeriod=30
Jul 23 12:50:01.235263 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.235236    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:50:01.324862 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.324825    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b40cafa-09e8-454b-a52d-3873b6c42296-openshift-service-ca\") pod \"4b40cafa-09e8-454b-a52d-3873b6c42296\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") "
Jul 23 12:50:01.324862 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.324867    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b40cafa-09e8-454b-a52d-3873b6c42296-maas-api-tls\") pod \"4b40cafa-09e8-454b-a52d-3873b6c42296\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") "
Jul 23 12:50:01.325100 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.324949    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skcqb\" (UniqueName: \"kubernetes.io/projected/4b40cafa-09e8-454b-a52d-3873b6c42296-kube-api-access-skcqb\") pod \"4b40cafa-09e8-454b-a52d-3873b6c42296\" (UID: \"4b40cafa-09e8-454b-a52d-3873b6c42296\") "
Jul 23 12:50:01.325224 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.325199    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b40cafa-09e8-454b-a52d-3873b6c42296-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b40cafa-09e8-454b-a52d-3873b6c42296" (UID: "4b40cafa-09e8-454b-a52d-3873b6c42296"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:50:01.327043 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.327017    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b40cafa-09e8-454b-a52d-3873b6c42296-kube-api-access-skcqb" (OuterVolumeSpecName: "kube-api-access-skcqb") pod "4b40cafa-09e8-454b-a52d-3873b6c42296" (UID: "4b40cafa-09e8-454b-a52d-3873b6c42296"). InnerVolumeSpecName "kube-api-access-skcqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:50:01.327135 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.327038    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b40cafa-09e8-454b-a52d-3873b6c42296-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b40cafa-09e8-454b-a52d-3873b6c42296" (UID: "4b40cafa-09e8-454b-a52d-3873b6c42296"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:50:01.425686 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.425648    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skcqb\" (UniqueName: \"kubernetes.io/projected/4b40cafa-09e8-454b-a52d-3873b6c42296-kube-api-access-skcqb\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:50:01.425686 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.425678    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b40cafa-09e8-454b-a52d-3873b6c42296-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:50:01.425686 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.425687    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b40cafa-09e8-454b-a52d-3873b6c42296-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:50:01.962664 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.962628    2565 generic.go:356] "Generic (PLEG): container finished" podID="4b40cafa-09e8-454b-a52d-3873b6c42296" containerID="bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e" exitCode=0
Jul 23 12:50:01.962882 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.962672    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"
Jul 23 12:50:01.962882 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.962712    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt" event={"ID":"4b40cafa-09e8-454b-a52d-3873b6c42296","Type":"ContainerDied","Data":"bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e"}
Jul 23 12:50:01.962882 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.962786    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt" event={"ID":"4b40cafa-09e8-454b-a52d-3873b6c42296","Type":"ContainerDied","Data":"4848fef357cea07b7e370d0d9768b157c7057dac92e4e6abfe6148db4b96dcb9"}
Jul 23 12:50:01.962882 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.962810    2565 scope.go:117] "RemoveContainer" containerID="bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e"
Jul 23 12:50:01.971590 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.971573    2565 scope.go:117] "RemoveContainer" containerID="bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e"
Jul 23 12:50:01.971871 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:50:01.971855    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e\": container with ID starting with bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e not found: ID does not exist" containerID="bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e"
Jul 23 12:50:01.971917 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.971880    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e"} err="failed to get container status \"bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e\": rpc error: code = NotFound desc = could not find container \"bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e\": container with ID starting with bd1856386de888736b3eb2fa4d015d7cb81d7067ecdab136b43f8a55c5ec727e not found: ID does not exist"
Jul 23 12:50:01.986495 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.986466    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"]
Jul 23 12:50:01.991803 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:01.991775    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-52006d-6d6dcd9cf8-g57mt"]
Jul 23 12:50:03.049911 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:03.049875    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b40cafa-09e8-454b-a52d-3873b6c42296" path="/var/lib/kubelet/pods/4b40cafa-09e8-454b-a52d-3873b6c42296/volumes"
Jul 23 12:50:05.619987 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:05.619949    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"]
Jul 23 12:50:05.623229 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:05.623197    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-52006d-x5jpq"]
Jul 23 12:50:07.050503 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:07.050471    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c208711-96c8-4f6d-b96c-4198cfa62f22" path="/var/lib/kubelet/pods/8c208711-96c8-4f6d-b96c-4198cfa62f22/volumes"
Jul 23 12:50:16.199187 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.199151    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"]
Jul 23 12:50:16.199725 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.199705    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b40cafa-09e8-454b-a52d-3873b6c42296" containerName="maas-api"
Jul 23 12:50:16.199806 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.199729    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b40cafa-09e8-454b-a52d-3873b6c42296" containerName="maas-api"
Jul 23 12:50:16.199806 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.199748    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c208711-96c8-4f6d-b96c-4198cfa62f22" containerName="revoke-keys"
Jul 23 12:50:16.199806 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.199777    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c208711-96c8-4f6d-b96c-4198cfa62f22" containerName="revoke-keys"
Jul 23 12:50:16.199906 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.199882    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c208711-96c8-4f6d-b96c-4198cfa62f22" containerName="revoke-keys"
Jul 23 12:50:16.199906 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.199895    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b40cafa-09e8-454b-a52d-3873b6c42296" containerName="maas-api"
Jul 23 12:50:16.204419 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.204399    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.207064 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.207040    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d45fc5-serving-cert\""
Jul 23 12:50:16.213836 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.213809    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"]
Jul 23 12:50:16.248858 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.248815    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdklx\" (UniqueName: \"kubernetes.io/projected/35690f8c-dd7b-4f31-86fe-a3c80caaa332-kube-api-access-wdklx\") pod \"maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.249114 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.248911    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35690f8c-dd7b-4f31-86fe-a3c80caaa332-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.249114 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.248966    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35690f8c-dd7b-4f31-86fe-a3c80caaa332-maas-api-tls\") pod \"maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.350234 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.350202    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdklx\" (UniqueName: \"kubernetes.io/projected/35690f8c-dd7b-4f31-86fe-a3c80caaa332-kube-api-access-wdklx\") pod \"maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.350410 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.350261    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35690f8c-dd7b-4f31-86fe-a3c80caaa332-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.350410 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.350292    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35690f8c-dd7b-4f31-86fe-a3c80caaa332-maas-api-tls\") pod \"maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.350889 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.350860    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35690f8c-dd7b-4f31-86fe-a3c80caaa332-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.352638 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.352612    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35690f8c-dd7b-4f31-86fe-a3c80caaa332-maas-api-tls\") pod \"maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.359599 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.359580    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdklx\" (UniqueName: \"kubernetes.io/projected/35690f8c-dd7b-4f31-86fe-a3c80caaa332-kube-api-access-wdklx\") pod \"maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.517007 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.516913    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:16.653652 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:16.653628    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"]
Jul 23 12:50:16.656291 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:50:16.656261    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35690f8c_dd7b_4f31_86fe_a3c80caaa332.slice/crio-557b0c2cf0f11aeb6836a8e31ef685f44e7186f63743ed353a8e45ac195f7f2e WatchSource:0}: Error finding container 557b0c2cf0f11aeb6836a8e31ef685f44e7186f63743ed353a8e45ac195f7f2e: Status 404 returned error can't find the container with id 557b0c2cf0f11aeb6836a8e31ef685f44e7186f63743ed353a8e45ac195f7f2e
Jul 23 12:50:17.060313 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:17.060266    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg" event={"ID":"35690f8c-dd7b-4f31-86fe-a3c80caaa332","Type":"ContainerStarted","Data":"557b0c2cf0f11aeb6836a8e31ef685f44e7186f63743ed353a8e45ac195f7f2e"}
Jul 23 12:50:18.061860 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:18.061822    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg" event={"ID":"35690f8c-dd7b-4f31-86fe-a3c80caaa332","Type":"ContainerStarted","Data":"78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc"}
Jul 23 12:50:18.062252 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:18.061893    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:18.080247 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:18.080196    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg" podStartSLOduration=1.747560931 podStartE2EDuration="2.080180325s" totalImagesPullingTime="332.619394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:50:18.079054062 +0000 UTC m=+2401.734562866" watchObservedRunningTime="2026-07-23 12:50:18.080180325 +0000 UTC m=+2401.735689129"
Jul 23 12:50:24.072502 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.072471    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:50:24.754733 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.754698    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"]
Jul 23 12:50:24.758612 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.758589    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:24.761298 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.761276    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-474ecb-serving-cert\""
Jul 23 12:50:24.769711 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.769681    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"]
Jul 23 12:50:24.822723 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.822691    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e98917c2-fc51-41cb-aa3f-e1a064afc420-maas-api-tls\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:24.822907 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.822782    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl2pc\" (UniqueName: \"kubernetes.io/projected/e98917c2-fc51-41cb-aa3f-e1a064afc420-kube-api-access-dl2pc\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:24.822907 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.822854    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e98917c2-fc51-41cb-aa3f-e1a064afc420-openshift-service-ca\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:24.923561 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.923516    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl2pc\" (UniqueName: \"kubernetes.io/projected/e98917c2-fc51-41cb-aa3f-e1a064afc420-kube-api-access-dl2pc\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:24.923561 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.923570    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e98917c2-fc51-41cb-aa3f-e1a064afc420-openshift-service-ca\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:24.923828 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.923606    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e98917c2-fc51-41cb-aa3f-e1a064afc420-maas-api-tls\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:24.923828 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:50:24.923702    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-serving-cert: secret "maas-api-e2e-shared-b-474ecb-serving-cert" not found
Jul 23 12:50:24.923828 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:50:24.923786    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e98917c2-fc51-41cb-aa3f-e1a064afc420-maas-api-tls podName:e98917c2-fc51-41cb-aa3f-e1a064afc420 nodeName:}" failed. No retries permitted until 2026-07-23 12:50:25.423742923 +0000 UTC m=+2409.079251706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e98917c2-fc51-41cb-aa3f-e1a064afc420-maas-api-tls") pod "maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l" (UID: "e98917c2-fc51-41cb-aa3f-e1a064afc420") : secret "maas-api-e2e-shared-b-474ecb-serving-cert" not found
Jul 23 12:50:24.924352 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.924331    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e98917c2-fc51-41cb-aa3f-e1a064afc420-openshift-service-ca\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:24.932686 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:24.932660    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl2pc\" (UniqueName: \"kubernetes.io/projected/e98917c2-fc51-41cb-aa3f-e1a064afc420-kube-api-access-dl2pc\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:25.428855 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:25.428819    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e98917c2-fc51-41cb-aa3f-e1a064afc420-maas-api-tls\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:25.431366 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:25.431342    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e98917c2-fc51-41cb-aa3f-e1a064afc420-maas-api-tls\") pod \"maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:25.670279 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:25.670239    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:25.810366 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:25.810334    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"]
Jul 23 12:50:25.813553 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:50:25.813524    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode98917c2_fc51_41cb_aa3f_e1a064afc420.slice/crio-c38f5133782e5d1bead87fefca098b27dcbddf47b6e98d05a080f5309ec12606 WatchSource:0}: Error finding container c38f5133782e5d1bead87fefca098b27dcbddf47b6e98d05a080f5309ec12606: Status 404 returned error can't find the container with id c38f5133782e5d1bead87fefca098b27dcbddf47b6e98d05a080f5309ec12606
Jul 23 12:50:26.103134 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:26.103094    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l" event={"ID":"e98917c2-fc51-41cb-aa3f-e1a064afc420","Type":"ContainerStarted","Data":"c38f5133782e5d1bead87fefca098b27dcbddf47b6e98d05a080f5309ec12606"}
Jul 23 12:50:26.798667 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:26.798638    2565 scope.go:117] "RemoveContainer" containerID="4a659ef4db42ca873a02b0a9f41fe64a974223052e80565cf820fd19346510ae"
Jul 23 12:50:27.109608 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:27.109573    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l" event={"ID":"e98917c2-fc51-41cb-aa3f-e1a064afc420","Type":"ContainerStarted","Data":"3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff"}
Jul 23 12:50:27.109833 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:27.109699    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:50:27.132395 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:27.132340    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l" podStartSLOduration=2.783097598 podStartE2EDuration="3.132321442s" totalImagesPullingTime="349.223844ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:50:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:50:27.130008755 +0000 UTC m=+2410.785517561" watchObservedRunningTime="2026-07-23 12:50:27.132321442 +0000 UTC m=+2410.787830245"
Jul 23 12:50:33.120509 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:50:33.120477    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:54:26.850340 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:54:26.850301    2565 scope.go:117] "RemoveContainer" containerID="ca85f33988f10127d66fe099d40c2b737f0f52fc4117cff163afdac70ad8707f"
Jul 23 12:55:26.874498 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:55:26.874464    2565 scope.go:117] "RemoveContainer" containerID="3fca467526f7e18494825fb53aa07059c2dd08cd24ad75812951f91553563ced"
Jul 23 12:55:26.882807 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:55:26.882789    2565 scope.go:117] "RemoveContainer" containerID="08879d754b274f87c5d84289b42340d90f006f0074f60440b19bf689cb9f3f10"
Jul 23 12:56:05.695612 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:05.695567    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7583-140481949798576"]
Jul 23 12:56:05.699119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:05.699102    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949798576"
Jul 23 12:56:05.701808 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:05.701789    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-srl6f\""
Jul 23 12:56:05.705743 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:05.705717    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7583-140481949798576"]
Jul 23 12:56:05.757887 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:05.757856    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbnm7\" (UniqueName: \"kubernetes.io/projected/7c03398f-d89d-408f-abcf-06b4cc94405c-kube-api-access-gbnm7\") pod \"test-curl-7583-140481949798576\" (UID: \"7c03398f-d89d-408f-abcf-06b4cc94405c\") " pod="opendatahub/test-curl-7583-140481949798576"
Jul 23 12:56:05.858950 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:05.858921    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbnm7\" (UniqueName: \"kubernetes.io/projected/7c03398f-d89d-408f-abcf-06b4cc94405c-kube-api-access-gbnm7\") pod \"test-curl-7583-140481949798576\" (UID: \"7c03398f-d89d-408f-abcf-06b4cc94405c\") " pod="opendatahub/test-curl-7583-140481949798576"
Jul 23 12:56:05.868176 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:05.868146    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbnm7\" (UniqueName: \"kubernetes.io/projected/7c03398f-d89d-408f-abcf-06b4cc94405c-kube-api-access-gbnm7\") pod \"test-curl-7583-140481949798576\" (UID: \"7c03398f-d89d-408f-abcf-06b4cc94405c\") " pod="opendatahub/test-curl-7583-140481949798576"
Jul 23 12:56:06.008867 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:06.008754    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949798576"
Jul 23 12:56:06.149324 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:06.149297    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7583-140481949798576"]
Jul 23 12:56:06.151206 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:56:06.151173    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c03398f_d89d_408f_abcf_06b4cc94405c.slice/crio-86f851f46f9ae173f6c80148e0b5edac85f28ce8a9b98a9b633426d95a7ef76c WatchSource:0}: Error finding container 86f851f46f9ae173f6c80148e0b5edac85f28ce8a9b98a9b633426d95a7ef76c: Status 404 returned error can't find the container with id 86f851f46f9ae173f6c80148e0b5edac85f28ce8a9b98a9b633426d95a7ef76c
Jul 23 12:56:06.152568 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:06.152546    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 12:56:06.737902 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:06.737856    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7583-140481949798576" event={"ID":"7c03398f-d89d-408f-abcf-06b4cc94405c","Type":"ContainerStarted","Data":"86f851f46f9ae173f6c80148e0b5edac85f28ce8a9b98a9b633426d95a7ef76c"}
Jul 23 12:56:07.743942 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:07.743903    2565 generic.go:356] "Generic (PLEG): container finished" podID="7c03398f-d89d-408f-abcf-06b4cc94405c" containerID="ea4238e596a20f35b2a5112bcd5a40651a830d9cc786bcb27d3711494157ff66" exitCode=0
Jul 23 12:56:07.744385 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:07.743992    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7583-140481949798576" event={"ID":"7c03398f-d89d-408f-abcf-06b4cc94405c","Type":"ContainerDied","Data":"ea4238e596a20f35b2a5112bcd5a40651a830d9cc786bcb27d3711494157ff66"}
Jul 23 12:56:08.886666 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:08.886641    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949798576"
Jul 23 12:56:08.925350 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:08.925322    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7583-140481949798576_7c03398f-d89d-408f-abcf-06b4cc94405c/test-curl-7583-140481949798576/0.log"
Jul 23 12:56:08.948908 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:08.948877    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7583-140481949798576"]
Jul 23 12:56:08.953393 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:08.953359    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7583-140481949798576"]
Jul 23 12:56:08.987848 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:08.987814    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbnm7\" (UniqueName: \"kubernetes.io/projected/7c03398f-d89d-408f-abcf-06b4cc94405c-kube-api-access-gbnm7\") pod \"7c03398f-d89d-408f-abcf-06b4cc94405c\" (UID: \"7c03398f-d89d-408f-abcf-06b4cc94405c\") "
Jul 23 12:56:08.990043 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:08.990012    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c03398f-d89d-408f-abcf-06b4cc94405c-kube-api-access-gbnm7" (OuterVolumeSpecName: "kube-api-access-gbnm7") pod "7c03398f-d89d-408f-abcf-06b4cc94405c" (UID: "7c03398f-d89d-408f-abcf-06b4cc94405c"). InnerVolumeSpecName "kube-api-access-gbnm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:56:09.050528 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:09.050451    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c03398f-d89d-408f-abcf-06b4cc94405c" path="/var/lib/kubelet/pods/7c03398f-d89d-408f-abcf-06b4cc94405c/volumes"
Jul 23 12:56:09.089206 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:09.089178    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbnm7\" (UniqueName: \"kubernetes.io/projected/7c03398f-d89d-408f-abcf-06b4cc94405c-kube-api-access-gbnm7\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:09.754708 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:09.754678    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949798576"
Jul 23 12:56:09.754936 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:09.754720    2565 scope.go:117] "RemoveContainer" containerID="ea4238e596a20f35b2a5112bcd5a40651a830d9cc786bcb27d3711494157ff66"
Jul 23 12:56:12.049832 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.049790    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:12.050331 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.050154    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c03398f-d89d-408f-abcf-06b4cc94405c" containerName="test-curl-7583-140481949798576"
Jul 23 12:56:12.050331 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.050167    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c03398f-d89d-408f-abcf-06b4cc94405c" containerName="test-curl-7583-140481949798576"
Jul 23 12:56:12.050331 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.050231    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c03398f-d89d-408f-abcf-06b4cc94405c" containerName="test-curl-7583-140481949798576"
Jul 23 12:56:12.056905 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.056862    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:12.059526 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.059505    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-srl6f\""
Jul 23 12:56:12.060874 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.060848    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:12.118583 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.118546    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w2hg\" (UniqueName: \"kubernetes.io/projected/234d5ff3-6d9f-4a4f-b0f6-3866667c26e9-kube-api-access-7w2hg\") pod \"test-curl-7583-140481949652000\" (UID: \"234d5ff3-6d9f-4a4f-b0f6-3866667c26e9\") " pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:12.219698 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.219659    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w2hg\" (UniqueName: \"kubernetes.io/projected/234d5ff3-6d9f-4a4f-b0f6-3866667c26e9-kube-api-access-7w2hg\") pod \"test-curl-7583-140481949652000\" (UID: \"234d5ff3-6d9f-4a4f-b0f6-3866667c26e9\") " pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:12.229952 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.229924    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w2hg\" (UniqueName: \"kubernetes.io/projected/234d5ff3-6d9f-4a4f-b0f6-3866667c26e9-kube-api-access-7w2hg\") pod \"test-curl-7583-140481949652000\" (UID: \"234d5ff3-6d9f-4a4f-b0f6-3866667c26e9\") " pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:12.367818 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.367743    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:12.505669 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.505645    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:12.507465 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:56:12.507441    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod234d5ff3_6d9f_4a4f_b0f6_3866667c26e9.slice/crio-c9516ce3396a2bbefbbfee828d1bdf6b155b062e3bbde93560934390fe6474b7 WatchSource:0}: Error finding container c9516ce3396a2bbefbbfee828d1bdf6b155b062e3bbde93560934390fe6474b7: Status 404 returned error can't find the container with id c9516ce3396a2bbefbbfee828d1bdf6b155b062e3bbde93560934390fe6474b7
Jul 23 12:56:12.773402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:12.773321    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7583-140481949652000" event={"ID":"234d5ff3-6d9f-4a4f-b0f6-3866667c26e9","Type":"ContainerStarted","Data":"c9516ce3396a2bbefbbfee828d1bdf6b155b062e3bbde93560934390fe6474b7"}
Jul 23 12:56:13.779453 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:13.779416    2565 generic.go:356] "Generic (PLEG): container finished" podID="234d5ff3-6d9f-4a4f-b0f6-3866667c26e9" containerID="e6c79742c16dc284ecffda3d8c9d3e4be2602c7659e5285df6a256c069ccc213" exitCode=0
Jul 23 12:56:13.779871 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:13.779460    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7583-140481949652000" event={"ID":"234d5ff3-6d9f-4a4f-b0f6-3866667c26e9","Type":"ContainerDied","Data":"e6c79742c16dc284ecffda3d8c9d3e4be2602c7659e5285df6a256c069ccc213"}
Jul 23 12:56:14.921096 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:14.921075    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:14.949455 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:14.949427    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7583-140481949652000_234d5ff3-6d9f-4a4f-b0f6-3866667c26e9/test-curl-7583-140481949652000/0.log"
Jul 23 12:56:14.988716 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:14.988679    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:14.992531 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:14.992507    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:15.046400 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.046317    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w2hg\" (UniqueName: \"kubernetes.io/projected/234d5ff3-6d9f-4a4f-b0f6-3866667c26e9-kube-api-access-7w2hg\") pod \"234d5ff3-6d9f-4a4f-b0f6-3866667c26e9\" (UID: \"234d5ff3-6d9f-4a4f-b0f6-3866667c26e9\") "
Jul 23 12:56:15.048429 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.048406    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/234d5ff3-6d9f-4a4f-b0f6-3866667c26e9-kube-api-access-7w2hg" (OuterVolumeSpecName: "kube-api-access-7w2hg") pod "234d5ff3-6d9f-4a4f-b0f6-3866667c26e9" (UID: "234d5ff3-6d9f-4a4f-b0f6-3866667c26e9"). InnerVolumeSpecName "kube-api-access-7w2hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:56:15.050119 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.050094    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="234d5ff3-6d9f-4a4f-b0f6-3866667c26e9" path="/var/lib/kubelet/pods/234d5ff3-6d9f-4a4f-b0f6-3866667c26e9/volumes"
Jul 23 12:56:15.119315 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.119280    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:15.119648 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.119635    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="234d5ff3-6d9f-4a4f-b0f6-3866667c26e9" containerName="test-curl-7583-140481949652000"
Jul 23 12:56:15.119688 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.119649    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="234d5ff3-6d9f-4a4f-b0f6-3866667c26e9" containerName="test-curl-7583-140481949652000"
Jul 23 12:56:15.119723 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.119702    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="234d5ff3-6d9f-4a4f-b0f6-3866667c26e9" containerName="test-curl-7583-140481949652000"
Jul 23 12:56:15.122591 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.122573    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:15.128397 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.128369    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:15.147062 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.147037    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w2hg\" (UniqueName: \"kubernetes.io/projected/234d5ff3-6d9f-4a4f-b0f6-3866667c26e9-kube-api-access-7w2hg\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:15.248308 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.248271    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qccdr\" (UniqueName: \"kubernetes.io/projected/426d646c-4773-4e83-b7df-537fcc98e118-kube-api-access-qccdr\") pod \"test-curl-7583-140481949652000\" (UID: \"426d646c-4773-4e83-b7df-537fcc98e118\") " pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:15.349529 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.349491    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qccdr\" (UniqueName: \"kubernetes.io/projected/426d646c-4773-4e83-b7df-537fcc98e118-kube-api-access-qccdr\") pod \"test-curl-7583-140481949652000\" (UID: \"426d646c-4773-4e83-b7df-537fcc98e118\") " pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:15.358628 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.358595    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qccdr\" (UniqueName: \"kubernetes.io/projected/426d646c-4773-4e83-b7df-537fcc98e118-kube-api-access-qccdr\") pod \"test-curl-7583-140481949652000\" (UID: \"426d646c-4773-4e83-b7df-537fcc98e118\") " pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:15.432621 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.432582    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:15.574592 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.574568    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:15.577085 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:56:15.577058    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod426d646c_4773_4e83_b7df_537fcc98e118.slice/crio-33336de18bbc75caa519137732331dd12be611824b032c8c02beb51423cb82df WatchSource:0}: Error finding container 33336de18bbc75caa519137732331dd12be611824b032c8c02beb51423cb82df: Status 404 returned error can't find the container with id 33336de18bbc75caa519137732331dd12be611824b032c8c02beb51423cb82df
Jul 23 12:56:15.789989 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.789948    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7583-140481949652000" event={"ID":"426d646c-4773-4e83-b7df-537fcc98e118","Type":"ContainerStarted","Data":"33336de18bbc75caa519137732331dd12be611824b032c8c02beb51423cb82df"}
Jul 23 12:56:15.791271 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.791245    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:15.791402 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:15.791250    2565 scope.go:117] "RemoveContainer" containerID="e6c79742c16dc284ecffda3d8c9d3e4be2602c7659e5285df6a256c069ccc213"
Jul 23 12:56:16.798289 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:16.798250    2565 generic.go:356] "Generic (PLEG): container finished" podID="426d646c-4773-4e83-b7df-537fcc98e118" containerID="1908581b793ab0bb14d2935504238d1daa884527fde5440071b71fb4f5df26ce" exitCode=0
Jul 23 12:56:16.798672 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:16.798340    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7583-140481949652000" event={"ID":"426d646c-4773-4e83-b7df-537fcc98e118","Type":"ContainerDied","Data":"1908581b793ab0bb14d2935504238d1daa884527fde5440071b71fb4f5df26ce"}
Jul 23 12:56:17.938886 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:17.938862    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:17.960269 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:17.960232    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7583-140481949652000_426d646c-4773-4e83-b7df-537fcc98e118/test-curl-7583-140481949652000/0.log"
Jul 23 12:56:17.983802 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:17.983752    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:17.986557 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:17.986530    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7583-140481949652000"]
Jul 23 12:56:18.079960 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:18.079928    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qccdr\" (UniqueName: \"kubernetes.io/projected/426d646c-4773-4e83-b7df-537fcc98e118-kube-api-access-qccdr\") pod \"426d646c-4773-4e83-b7df-537fcc98e118\" (UID: \"426d646c-4773-4e83-b7df-537fcc98e118\") "
Jul 23 12:56:18.082192 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:18.082160    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/426d646c-4773-4e83-b7df-537fcc98e118-kube-api-access-qccdr" (OuterVolumeSpecName: "kube-api-access-qccdr") pod "426d646c-4773-4e83-b7df-537fcc98e118" (UID: "426d646c-4773-4e83-b7df-537fcc98e118"). InnerVolumeSpecName "kube-api-access-qccdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:56:18.180718 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:18.180677    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qccdr\" (UniqueName: \"kubernetes.io/projected/426d646c-4773-4e83-b7df-537fcc98e118-kube-api-access-qccdr\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:18.809778 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:18.809730    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33336de18bbc75caa519137732331dd12be611824b032c8c02beb51423cb82df"
Jul 23 12:56:18.809778 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:18.809733    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:18.819730 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:56:18.819697    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7583-140481949652000\" is forbidden: User \"system:node:ip-10-0-137-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-16.ec2.internal' and this object" podUID="426d646c-4773-4e83-b7df-537fcc98e118" pod="opendatahub/test-curl-7583-140481949652000"
Jul 23 12:56:19.049691 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:19.049660    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="426d646c-4773-4e83-b7df-537fcc98e118" path="/var/lib/kubelet/pods/426d646c-4773-4e83-b7df-537fcc98e118/volumes"
Jul 23 12:56:26.903712 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:26.903680    2565 scope.go:117] "RemoveContainer" containerID="751cc0d5c3f0fc1ca9e77534ec6ae5c02c80d7fe5bc9f88625bd609202499633"
Jul 23 12:56:34.587585 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.587548    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"]
Jul 23 12:56:34.588069 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.587918    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg" podUID="35690f8c-dd7b-4f31-86fe-a3c80caaa332" containerName="maas-api" containerID="cri-o://78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc" gracePeriod=30
Jul 23 12:56:34.844534 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.844463    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:56:34.886381 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.886349    2565 generic.go:356] "Generic (PLEG): container finished" podID="35690f8c-dd7b-4f31-86fe-a3c80caaa332" containerID="78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc" exitCode=0
Jul 23 12:56:34.886529 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.886397    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"
Jul 23 12:56:34.886529 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.886434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg" event={"ID":"35690f8c-dd7b-4f31-86fe-a3c80caaa332","Type":"ContainerDied","Data":"78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc"}
Jul 23 12:56:34.886529 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.886494    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg" event={"ID":"35690f8c-dd7b-4f31-86fe-a3c80caaa332","Type":"ContainerDied","Data":"557b0c2cf0f11aeb6836a8e31ef685f44e7186f63743ed353a8e45ac195f7f2e"}
Jul 23 12:56:34.886529 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.886517    2565 scope.go:117] "RemoveContainer" containerID="78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc"
Jul 23 12:56:34.894652 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.894621    2565 scope.go:117] "RemoveContainer" containerID="78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc"
Jul 23 12:56:34.894923 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:56:34.894904    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc\": container with ID starting with 78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc not found: ID does not exist" containerID="78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc"
Jul 23 12:56:34.894973 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.894933    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc"} err="failed to get container status \"78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc\": rpc error: code = NotFound desc = could not find container \"78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc\": container with ID starting with 78b688d9deb35afa6731623da9d64a8a1316137b4ec77e45a248203c450debdc not found: ID does not exist"
Jul 23 12:56:34.920782 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.920731    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdklx\" (UniqueName: \"kubernetes.io/projected/35690f8c-dd7b-4f31-86fe-a3c80caaa332-kube-api-access-wdklx\") pod \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") "
Jul 23 12:56:34.920927 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.920796    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35690f8c-dd7b-4f31-86fe-a3c80caaa332-maas-api-tls\") pod \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") "
Jul 23 12:56:34.920927 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.920823    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35690f8c-dd7b-4f31-86fe-a3c80caaa332-openshift-service-ca\") pod \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\" (UID: \"35690f8c-dd7b-4f31-86fe-a3c80caaa332\") "
Jul 23 12:56:34.921236 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.921210    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35690f8c-dd7b-4f31-86fe-a3c80caaa332-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35690f8c-dd7b-4f31-86fe-a3c80caaa332" (UID: "35690f8c-dd7b-4f31-86fe-a3c80caaa332"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:56:34.922881 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.922861    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35690f8c-dd7b-4f31-86fe-a3c80caaa332-kube-api-access-wdklx" (OuterVolumeSpecName: "kube-api-access-wdklx") pod "35690f8c-dd7b-4f31-86fe-a3c80caaa332" (UID: "35690f8c-dd7b-4f31-86fe-a3c80caaa332"). InnerVolumeSpecName "kube-api-access-wdklx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:56:34.922983 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:34.922962    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35690f8c-dd7b-4f31-86fe-a3c80caaa332-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35690f8c-dd7b-4f31-86fe-a3c80caaa332" (UID: "35690f8c-dd7b-4f31-86fe-a3c80caaa332"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:56:35.021943 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:35.021894    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdklx\" (UniqueName: \"kubernetes.io/projected/35690f8c-dd7b-4f31-86fe-a3c80caaa332-kube-api-access-wdklx\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:35.021943 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:35.021939    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35690f8c-dd7b-4f31-86fe-a3c80caaa332-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:35.021943 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:35.021949    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35690f8c-dd7b-4f31-86fe-a3c80caaa332-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:35.204248 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:35.204170    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"]
Jul 23 12:56:35.209489 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:35.209463    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d45fc5-94dd6cd58-d5stg"]
Jul 23 12:56:37.049625 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:37.049591    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35690f8c-dd7b-4f31-86fe-a3c80caaa332" path="/var/lib/kubelet/pods/35690f8c-dd7b-4f31-86fe-a3c80caaa332/volumes"
Jul 23 12:56:46.594175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.591449    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"]
Jul 23 12:56:46.594175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.592265    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35690f8c-dd7b-4f31-86fe-a3c80caaa332" containerName="maas-api"
Jul 23 12:56:46.594175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.592285    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="35690f8c-dd7b-4f31-86fe-a3c80caaa332" containerName="maas-api"
Jul 23 12:56:46.594175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.592324    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426d646c-4773-4e83-b7df-537fcc98e118" containerName="test-curl-7583-140481949652000"
Jul 23 12:56:46.594175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.592340    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="426d646c-4773-4e83-b7df-537fcc98e118" containerName="test-curl-7583-140481949652000"
Jul 23 12:56:46.594175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.592513    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="35690f8c-dd7b-4f31-86fe-a3c80caaa332" containerName="maas-api"
Jul 23 12:56:46.594175 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.592534    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="426d646c-4773-4e83-b7df-537fcc98e118" containerName="test-curl-7583-140481949652000"
Jul 23 12:56:46.598789 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.598496    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"
Jul 23 12:56:46.600836 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.600810    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"]
Jul 23 12:56:46.601515 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.601492    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\""
Jul 23 12:56:46.725821 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.725755    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e488b387-3865-4a86-a5f4-11d907c42329-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp\" (UID: \"e488b387-3865-4a86-a5f4-11d907c42329\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"
Jul 23 12:56:46.826911 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.826875    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e488b387-3865-4a86-a5f4-11d907c42329-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp\" (UID: \"e488b387-3865-4a86-a5f4-11d907c42329\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"
Jul 23 12:56:46.827486 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.827465    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e488b387-3865-4a86-a5f4-11d907c42329-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp\" (UID: \"e488b387-3865-4a86-a5f4-11d907c42329\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"
Jul 23 12:56:46.911329 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:46.911235    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"
Jul 23 12:56:47.053877 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:47.053851    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"]
Jul 23 12:56:47.054387 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:56:47.054360    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode488b387_3865_4a86_a5f4_11d907c42329.slice/crio-ee9e9a3beaef73162f2221aad80bb4e191984e0e971647dd4636a43fccddcb15 WatchSource:0}: Error finding container ee9e9a3beaef73162f2221aad80bb4e191984e0e971647dd4636a43fccddcb15: Status 404 returned error can't find the container with id ee9e9a3beaef73162f2221aad80bb4e191984e0e971647dd4636a43fccddcb15
Jul 23 12:56:47.949839 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:47.949802    2565 generic.go:356] "Generic (PLEG): container finished" podID="e488b387-3865-4a86-a5f4-11d907c42329" containerID="5f7fb88ab9473372a43843f3075f14f41c914383bb981bf7cc73f255eb99ca39" exitCode=0
Jul 23 12:56:47.950224 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:47.949867    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp" event={"ID":"e488b387-3865-4a86-a5f4-11d907c42329","Type":"ContainerDied","Data":"5f7fb88ab9473372a43843f3075f14f41c914383bb981bf7cc73f255eb99ca39"}
Jul 23 12:56:47.950224 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:47.949901    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp" event={"ID":"e488b387-3865-4a86-a5f4-11d907c42329","Type":"ContainerStarted","Data":"ee9e9a3beaef73162f2221aad80bb4e191984e0e971647dd4636a43fccddcb15"}
Jul 23 12:56:49.088574 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:49.088546    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"
Jul 23 12:56:49.146111 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:49.146068    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e488b387-3865-4a86-a5f4-11d907c42329-maas-api-service-ca\") pod \"e488b387-3865-4a86-a5f4-11d907c42329\" (UID: \"e488b387-3865-4a86-a5f4-11d907c42329\") "
Jul 23 12:56:49.146413 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:49.146388    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e488b387-3865-4a86-a5f4-11d907c42329-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e488b387-3865-4a86-a5f4-11d907c42329" (UID: "e488b387-3865-4a86-a5f4-11d907c42329"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:56:49.247226 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:49.247124    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e488b387-3865-4a86-a5f4-11d907c42329-maas-api-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:49.960159 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:49.960127    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"
Jul 23 12:56:49.960337 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:49.960131    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp" event={"ID":"e488b387-3865-4a86-a5f4-11d907c42329","Type":"ContainerDied","Data":"ee9e9a3beaef73162f2221aad80bb4e191984e0e971647dd4636a43fccddcb15"}
Jul 23 12:56:49.960337 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:49.960252    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee9e9a3beaef73162f2221aad80bb4e191984e0e971647dd4636a43fccddcb15"
Jul 23 12:56:57.166644 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.166606    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"]
Jul 23 12:56:57.167149 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.166878    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l" podUID="e98917c2-fc51-41cb-aa3f-e1a064afc420" containerName="maas-api" containerID="cri-o://3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff" gracePeriod=30
Jul 23 12:56:57.415915 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.415889    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:56:57.519689 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.519593    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl2pc\" (UniqueName: \"kubernetes.io/projected/e98917c2-fc51-41cb-aa3f-e1a064afc420-kube-api-access-dl2pc\") pod \"e98917c2-fc51-41cb-aa3f-e1a064afc420\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") "
Jul 23 12:56:57.519689 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.519648    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e98917c2-fc51-41cb-aa3f-e1a064afc420-openshift-service-ca\") pod \"e98917c2-fc51-41cb-aa3f-e1a064afc420\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") "
Jul 23 12:56:57.519953 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.519694    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e98917c2-fc51-41cb-aa3f-e1a064afc420-maas-api-tls\") pod \"e98917c2-fc51-41cb-aa3f-e1a064afc420\" (UID: \"e98917c2-fc51-41cb-aa3f-e1a064afc420\") "
Jul 23 12:56:57.520105 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.520074    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e98917c2-fc51-41cb-aa3f-e1a064afc420-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e98917c2-fc51-41cb-aa3f-e1a064afc420" (UID: "e98917c2-fc51-41cb-aa3f-e1a064afc420"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:56:57.521808 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.521777    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e98917c2-fc51-41cb-aa3f-e1a064afc420-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e98917c2-fc51-41cb-aa3f-e1a064afc420" (UID: "e98917c2-fc51-41cb-aa3f-e1a064afc420"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:56:57.521915 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.521881    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e98917c2-fc51-41cb-aa3f-e1a064afc420-kube-api-access-dl2pc" (OuterVolumeSpecName: "kube-api-access-dl2pc") pod "e98917c2-fc51-41cb-aa3f-e1a064afc420" (UID: "e98917c2-fc51-41cb-aa3f-e1a064afc420"). InnerVolumeSpecName "kube-api-access-dl2pc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:56:57.621181 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.621144    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e98917c2-fc51-41cb-aa3f-e1a064afc420-maas-api-tls\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:57.621181 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.621179    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl2pc\" (UniqueName: \"kubernetes.io/projected/e98917c2-fc51-41cb-aa3f-e1a064afc420-kube-api-access-dl2pc\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:57.621181 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.621191    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e98917c2-fc51-41cb-aa3f-e1a064afc420-openshift-service-ca\") on node \"ip-10-0-137-16.ec2.internal\" DevicePath \"\""
Jul 23 12:56:57.998217 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.998179    2565 generic.go:356] "Generic (PLEG): container finished" podID="e98917c2-fc51-41cb-aa3f-e1a064afc420" containerID="3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff" exitCode=0
Jul 23 12:56:57.998395 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.998229    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"
Jul 23 12:56:57.998395 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.998239    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l" event={"ID":"e98917c2-fc51-41cb-aa3f-e1a064afc420","Type":"ContainerDied","Data":"3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff"}
Jul 23 12:56:57.998395 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.998292    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l" event={"ID":"e98917c2-fc51-41cb-aa3f-e1a064afc420","Type":"ContainerDied","Data":"c38f5133782e5d1bead87fefca098b27dcbddf47b6e98d05a080f5309ec12606"}
Jul 23 12:56:57.998395 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:57.998322    2565 scope.go:117] "RemoveContainer" containerID="3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff"
Jul 23 12:56:58.007523 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:58.007502    2565 scope.go:117] "RemoveContainer" containerID="3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff"
Jul 23 12:56:58.007805 ip-10-0-137-16 kubenswrapper[2565]: E0723 12:56:58.007785    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff\": container with ID starting with 3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff not found: ID does not exist" containerID="3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff"
Jul 23 12:56:58.007884 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:58.007814    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff"} err="failed to get container status \"3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff\": rpc error: code = NotFound desc = could not find container \"3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff\": container with ID starting with 3f4300c29a158aa3441e994c4c3831bc80f6c52a74de0fb98ed3ad5d65d78dff not found: ID does not exist"
Jul 23 12:56:58.019938 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:58.019907    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"]
Jul 23 12:56:58.024101 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:58.024077    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-474ecb-6d4d78bbbf-mgd8l"]
Jul 23 12:56:59.050093 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:56:59.050057    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e98917c2-fc51-41cb-aa3f-e1a064afc420" path="/var/lib/kubelet/pods/e98917c2-fc51-41cb-aa3f-e1a064afc420/volumes"
Jul 23 12:57:01.670889 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:01.670848    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"]
Jul 23 12:57:01.673741 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:01.673712    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-474ecb-d4bpp"]
Jul 23 12:57:03.049492 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:03.049457    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e488b387-3865-4a86-a5f4-11d907c42329" path="/var/lib/kubelet/pods/e488b387-3865-4a86-a5f4-11d907c42329/volumes"
Jul 23 12:57:15.094263 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:15.094218    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xhlcv_c3544862-f020-4982-abbc-32788f9c8649/manager/0.log"
Jul 23 12:57:15.219955 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:15.219923    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-m7cmn_1aa67db6-d44a-4399-ba23-ce61cffb25ac/manager/0.log"
Jul 23 12:57:15.349160 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:15.349077    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-645984f55d-sdxx7_cf388995-0283-40ec-9ce5-5b989989e4ef/manager/0.log"
Jul 23 12:57:15.475256 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:15.475227    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4pptf_5e4c8a7e-873f-4d58-aa64-178cae6dd57d/server/0.log"
Jul 23 12:57:16.020940 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:16.020906    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b_52116d9e-e158-442e-a64c-a6e7b02d0973/copy-manifests/0.log"
Jul 23 12:57:16.027710 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:16.027674    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b_52116d9e-e158-442e-a64c-a6e7b02d0973/manager/0.log"
Jul 23 12:57:16.391954 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:16.391909    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-776cd7fd75-gtnpj_a5476f4d-df53-40ad-95d1-c737e8f209fa/maas-api/0.log"
Jul 23 12:57:16.507217 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:16.507191    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29746830-xn5g6_03fde351-87dd-4982-9c42-b35ad86c2512/cleanup/0.log"
Jul 23 12:57:16.751286 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:16.751197    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-fxlxl_2169ce02-42fb-4e1f-820d-0271b4086952/postgres/0.log"
Jul 23 12:57:18.284642 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:18.284598    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8nrc9_1d38df85-7e53-4ba1-bbf5-413590a41145/manager/0.log"
Jul 23 12:57:18.408792 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:18.408746    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nc2t9_3f87b170-6812-4707-bd46-87bf942c00af/manager/0.log"
Jul 23 12:57:18.529877 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:18.529849    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xt2t5_426d23b7-7724-4b27-8b50-b8e81e46de5b/kuadrant-console-plugin/0.log"
Jul 23 12:57:18.649366 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:18.649331    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mkjq8_a0a054f4-3278-425f-ada5-65f0d5b116f2/registry-server/0.log"
Jul 23 12:57:18.776917 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:18.776883    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vf4ht_1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e/manager/0.log"
Jul 23 12:57:19.024200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:19.024081    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-dzvlb_313fa7b5-afb2-4756-b983-19ca5536c3c4/manager/0.log"
Jul 23 12:57:19.647022 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:19.646992    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f7c78d7c4-j7pjt_fbc1ee4a-41f8-454d-9058-8035ca02e135/kube-auth-proxy/0.log"
Jul 23 12:57:20.741621 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:20.741594    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-4hwj9_20fd37de-6b45-4eae-ad1a-237c47e70305/main/0.log"
Jul 23 12:57:20.986739 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:20.986711    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9b7mx_6158f94b-9fa9-493f-b480-bfa562dbff56/main/0.log"
Jul 23 12:57:21.107623 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:21.107591    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rgb4j_5e478984-3678-4efa-92d2-abfeaaf77734/main/0.log"
Jul 23 12:57:29.720846 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:29.720815    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f22wm_aa53ff2f-84ce-4767-8e00-a0230504134b/global-pull-secret-syncer/0.log"
Jul 23 12:57:29.892720 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:29.892678    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7wjgn_44b339ea-0da4-414b-b554-25c5eb99bef7/konnectivity-agent/0.log"
Jul 23 12:57:29.978353 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:29.978270    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-16.ec2.internal_05afcdaf9e6abfaa255292cf98e82059/haproxy/0.log"
Jul 23 12:57:35.569110 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:35.569074    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8nrc9_1d38df85-7e53-4ba1-bbf5-413590a41145/manager/0.log"
Jul 23 12:57:35.603162 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:35.603132    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nc2t9_3f87b170-6812-4707-bd46-87bf942c00af/manager/0.log"
Jul 23 12:57:35.630946 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:35.630913    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xt2t5_426d23b7-7724-4b27-8b50-b8e81e46de5b/kuadrant-console-plugin/0.log"
Jul 23 12:57:35.692489 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:35.692459    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mkjq8_a0a054f4-3278-425f-ada5-65f0d5b116f2/registry-server/0.log"
Jul 23 12:57:35.826587 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:35.826488    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vf4ht_1f45b2a5-05b1-4c92-9aa2-40a2aaa9840e/manager/0.log"
Jul 23 12:57:35.924854 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:35.924810    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-dzvlb_313fa7b5-afb2-4756-b983-19ca5536c3c4/manager/0.log"
Jul 23 12:57:38.140838 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:38.140809    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6kfl_f918b10a-19fc-4a4e-a932-2631b9cd6c53/node-exporter/0.log"
Jul 23 12:57:38.164362 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:38.164330    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6kfl_f918b10a-19fc-4a4e-a932-2631b9cd6c53/kube-rbac-proxy/0.log"
Jul 23 12:57:38.189039 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:38.189003    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6kfl_f918b10a-19fc-4a4e-a932-2631b9cd6c53/init-textfile/0.log"
Jul 23 12:57:40.065928 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.065900    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-h2w84_3848ca6a-ae6a-470d-ad4f-b1cadc0fb629/networking-console-plugin/0.log"
Jul 23 12:57:40.296839 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.296802    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"]
Jul 23 12:57:40.297222 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.297205    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e488b387-3865-4a86-a5f4-11d907c42329" containerName="revoke-keys"
Jul 23 12:57:40.297222 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.297223    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e488b387-3865-4a86-a5f4-11d907c42329" containerName="revoke-keys"
Jul 23 12:57:40.297341 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.297242    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e98917c2-fc51-41cb-aa3f-e1a064afc420" containerName="maas-api"
Jul 23 12:57:40.297341 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.297248    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e98917c2-fc51-41cb-aa3f-e1a064afc420" containerName="maas-api"
Jul 23 12:57:40.297341 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.297300    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e98917c2-fc51-41cb-aa3f-e1a064afc420" containerName="maas-api"
Jul 23 12:57:40.297341 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.297311    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e488b387-3865-4a86-a5f4-11d907c42329" containerName="revoke-keys"
Jul 23 12:57:40.301598 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.301579    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.304225 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.304203    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jmcfw\"/\"openshift-service-ca.crt\""
Jul 23 12:57:40.304365 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.304352    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jmcfw\"/\"default-dockercfg-v244z\""
Jul 23 12:57:40.304454 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.304391    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jmcfw\"/\"kube-root-ca.crt\""
Jul 23 12:57:40.307984 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.307955    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"]
Jul 23 12:57:40.379104 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.379018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-proc\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.379104 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.379072    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-sys\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.379295 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.379122    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-lib-modules\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.379295 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.379150    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-podres\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.379295 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.379193    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95p2s\" (UniqueName: \"kubernetes.io/projected/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-kube-api-access-95p2s\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.480618 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.480573    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-proc\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.480826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.480630    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-sys\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.480826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.480652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-lib-modules\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.480826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.480667    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-podres\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.480826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.480681    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-proc\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.480826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.480685    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95p2s\" (UniqueName: \"kubernetes.io/projected/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-kube-api-access-95p2s\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.480826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.480749    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-sys\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.481100 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.480839    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-lib-modules\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.481100 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.480858    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-podres\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.490254 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.490222    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95p2s\" (UniqueName: \"kubernetes.io/projected/b2f9abc4-ecb6-43c7-a560-63ed1dc47c58-kube-api-access-95p2s\") pod \"perf-node-gather-daemonset-xr5rx\" (UID: \"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.612924 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.612884    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:40.745546 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:40.745512    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"]
Jul 23 12:57:40.749592 ip-10-0-137-16 kubenswrapper[2565]: W0723 12:57:40.749564    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb2f9abc4_ecb6_43c7_a560_63ed1dc47c58.slice/crio-c99fbabffa18ac772088e511c398bfa649fa573dbb277042760e452de68ec1cc WatchSource:0}: Error finding container c99fbabffa18ac772088e511c398bfa649fa573dbb277042760e452de68ec1cc: Status 404 returned error can't find the container with id c99fbabffa18ac772088e511c398bfa649fa573dbb277042760e452de68ec1cc
Jul 23 12:57:41.196672 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:41.196638    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx" event={"ID":"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58","Type":"ContainerStarted","Data":"d19a4149d5d4d61fbdd95e6b136173278e134394517969e3626c163508079c93"}
Jul 23 12:57:41.197200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:41.196696    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx" event={"ID":"b2f9abc4-ecb6-43c7-a560-63ed1dc47c58","Type":"ContainerStarted","Data":"c99fbabffa18ac772088e511c398bfa649fa573dbb277042760e452de68ec1cc"}
Jul 23 12:57:41.197200 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:41.196784    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:41.213716 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:41.213669    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx" podStartSLOduration=1.21365217 podStartE2EDuration="1.21365217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:57:41.212956469 +0000 UTC m=+2844.868465285" watchObservedRunningTime="2026-07-23 12:57:41.21365217 +0000 UTC m=+2844.869160974"
Jul 23 12:57:43.112387 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:43.112352    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xzfkw_4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b/dns/0.log"
Jul 23 12:57:43.136424 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:43.136392    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xzfkw_4e49b1de-e7f2-41f8-b9d1-e2a8a21e360b/kube-rbac-proxy/0.log"
Jul 23 12:57:43.160437 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:43.160412    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-89d2f_c24d092b-c557-43b9-a1ce-dd4d7eb8f55c/dns-node-resolver/0.log"
Jul 23 12:57:43.778118 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:43.778092    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9dtwz_52ac0e67-2df0-4d14-84b1-06ba64808ccf/node-ca/0.log"
Jul 23 12:57:45.099620 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:45.099596    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f7c78d7c4-j7pjt_fbc1ee4a-41f8-454d-9058-8035ca02e135/kube-auth-proxy/0.log"
Jul 23 12:57:45.944897 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:45.944871    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2xtcb_17da2583-078d-4eb3-88f2-640d28af40ba/serve-healthcheck-canary/0.log"
Jul 23 12:57:46.679948 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:46.679918    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vsx98_008cf23c-b927-426d-9a6a-e69d58a5ba3b/kube-rbac-proxy/0.log"
Jul 23 12:57:46.703344 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:46.703316    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vsx98_008cf23c-b927-426d-9a6a-e69d58a5ba3b/exporter/0.log"
Jul 23 12:57:46.727078 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:46.727028    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vsx98_008cf23c-b927-426d-9a6a-e69d58a5ba3b/extractor/0.log"
Jul 23 12:57:47.215532 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:47.215502    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-xr5rx"
Jul 23 12:57:48.964530 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:48.964487    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xhlcv_c3544862-f020-4982-abbc-32788f9c8649/manager/0.log"
Jul 23 12:57:49.148826 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:49.148712    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-m7cmn_1aa67db6-d44a-4399-ba23-ce61cffb25ac/manager/0.log"
Jul 23 12:57:49.265471 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:49.265437    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-645984f55d-sdxx7_cf388995-0283-40ec-9ce5-5b989989e4ef/manager/0.log"
Jul 23 12:57:49.287855 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:49.287833    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4pptf_5e4c8a7e-873f-4d58-aa64-178cae6dd57d/server/0.log"
Jul 23 12:57:49.576017 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:49.575535    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b_52116d9e-e158-442e-a64c-a6e7b02d0973/manager/0.log"
Jul 23 12:57:49.603686 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:49.603657    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d6ddd8c6d-vqg5b_52116d9e-e158-442e-a64c-a6e7b02d0973/copy-manifests/0.log"
Jul 23 12:57:50.891241 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:50.891214    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f7c78d7c4-j7pjt_fbc1ee4a-41f8-454d-9058-8035ca02e135/kube-auth-proxy/0.log"
Jul 23 12:57:52.425538 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:52.425509    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f7c78d7c4-j7pjt_fbc1ee4a-41f8-454d-9058-8035ca02e135/kube-auth-proxy/0.log"
Jul 23 12:57:57.625575 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:57.625543    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrgwq_17efe563-dcf8-4bf3-9c27-dbdecf122831/kube-multus-additional-cni-plugins/0.log"
Jul 23 12:57:57.648105 ip-10-0-137-16 kubenswrapper[2565]: I0723 12:57:57.648072    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrgwq_17efe563-dcf8-4bf3-9c27-dbdecf122831/egress-router-binary-copy/0.log"